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AMSAA  RELIABILITY  GROWTH  GUIDE 


1.  INTRODUCTION 

1.1  Foreword.  This  guide  provides  methodology  and  concepts  to  assist  in  reliability 
growth  planning  and  a  structured  approach  for  reliability  growth  assessments.  The 
planning  aspects,  which  are  covered  in  Section  2  of  this  guide,  address  the  planned 
CTOWth  curve  and  related  milestones.  The  assessment  techniques,  which  are  designed  to 
realistically  evaluate  reliability  in  the  presence  of  a  changing  configuration,  are  based  on 
demonstrated  and  projected  values  and  are  covered  m  Sections  3  and  4,  respectively.  The 
material  in  this  guide  updates  MIL-HDBK-189  [1]. 

1.1.1  Why.  Reliability  growth  management  procedures  were  developed  to  help 
guide  the  materiel  acquisition  process  for  new  military  systems.  This  process  is  usually 
complex  and  difficult  for  many  reasons.  Generally,  these  systems  require  new 
technologies  and  represent  a  challenge  to  the  state  of  the  art.  Moreover,  the  requirements 
for  reliability,  maintainability  and  other  performance  parameters  are  usually  hig  y 
demanding.  Consequently,  striving  to  meet  these  requirements  represents  a  signific^t 
portion  of  the  entire  acquisition  process  and,  as  a  result,  the  setting  of  pnonties  and  the 
allocation  and  reallocation  of  resources  such  as  funds,  manpower  and  time  are  often 
formidable  management  tasks. 

1.1.2  What.  Reliability  growth  management  procedures  address  the  priorities 
and  allocation  problem.  These  techniques  will  enable  the  manager  to  plan,  evaluate  and 
control  the  reliability  of  a  system  during  its  development  stage.  The  reliability  growt 
concepts  and  methodologies  presented  in  this  guide  have  evolved  over  Ae  last  couple  of 
decades  by  actual  applications  to  Army,  Navy  and  Air  Force  systems.  Through  the 
application  of  these  reliability  growth  techniques  significant  payoffs  can  be  realized  from 
the  resulting  effective  management  of  attaining  system  reliability  goals. 

1.1.3  Layout.  This  guide  is  written  for  both  the  manager  and  the  analyst. 

Generally,  the  further  into  the  guide  one  reads,  the  more  technical  and  detailed  the 
material  becomes.  The  fundamental  concepts  are  covered  early  m  the  guide  and  the 
details  regarding  the  implementation  of  these  concepts  are  discussed  pnmanly  m  the 
latter  sections.  This  format,  together  with  an  objective  for  as  much  completeness  as 
possible  within  each  section,  have  resulted  in  some  concepts  being  repeated  or  discussed 
in  more  than  one  place.  This  should  help  facilitate  the  use  of  the  guide  for  studying 
certain  topics  without  extensively  referring  to  previous  matenal. 

1.2  Scope. 

1.2.1  Purpose.  This  guide  provides  an  understanding  of  the  concepts  and 

principles  of  reliability  growth,  advantages  of  managing  reliability  growth,  and  guidelines 
and  procedures  to  be  used  in  managing  reliability  growth.  It  should  be  noted  that  this 
guide  is  not  intended  to  serve  as  a  reliability  growth  plan  to  be  applied  to  a  progr^ 
without  any  tailoring.  This  guide,  when  used  in  conjunction  with  knowledge  of  the 


rrrciTn  Will  allow  tlic  devclopment  of  a  reliability  growth 

1.3  Definition  of  Terms. 

1.3.1  ReliablUty.  Reliability  is  the 

intended  function  for  a  j  “^faescribed  in  a  mission  profile.  The  term 

“'"f  ttr;rde"  the  scenario  in  which  the  system 

wmt^^  Vse  conditions  should  reflect  operational  usage. 

process.  . 

1J.3  Reliability  Growth  Seller  resources, 

systematic  planning  for  '*'‘**^f by  reallocation  of  resources  based  on 

and  assessed  reliability  values. 

134  Repair.  Arepahisther^lacementofafaileditemwithan-hdenUcal 

item  in  order  to  return  the  item  to  its  mission. 

.Henianu«pr-s‘s%7S^^^ 

1 4  Overview. 

141  Benefits  of  Reliability  Growth  Management.  The  following  benefits  can 

be  realiL'  by  the  utilization  of  reliability  growth  management. 

1.4.1.1  Finding  Unforeseen  “;^^bly 

system  with  major  technological  adv^c  design  stage.  This  is  also 

performance  deficiencies  that  cou  integration  of  existing  systems.  Unforeseen 
true  of  prototypes  that  are  simply  interoperation  and  interfacing  between 

1.4.1.2  Designing  in  '“P™’® ‘”“^!‘g"^ce  subjected 

potential  problems  can  be  “f^h„se  problems  that  drive  the  failure  rate  so 

to  a  development  testing  P'^St™  “  “  ^  a*  ig„  can  be  made.  The  ensuing  system 
that  the  necessary  improvements  in  system  design 
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reliability  and  performance  requirements. 

p^rfiipinp  the  Risk  of  Final  Demonstration.  Experience  has  shown  that 

1.4.1. 3  Reducing  demonstration  by  itself  to  determine  compliance  with 

programs  that  rely  simply  on  a  fm  arhieve  the  reliability  objectives  within 

the  reliability  requirements  do  not,  m  many  cases,  achie 

chance  of  passing,  or  even  replace  a  final  demonstration. 

1414  Tncreasinc  the  Probability  of  Meeting  Objectives.  This  can  be 

achieved'  by  'setting  interim  reli^iU^  S^als. 

rc!^retnri:e”:prrSo  r growth  management  thronghont  the 
development  program  organizes  this  process. 

141  Sketch  of  Reliability  Growth  Management.  The  essence  of  reliability 

growth  man\gemTn^on^«  of  evaluating  and  controlling  the  growth  process. 

1421  Reliability  Growth  Planning.  Reliability  growth  plamiing  addresses 

prograt^'—s.  amou^  of  testing 

reliability  goals  throughout  the  program. 

1  4  2  2  Reliability  Growth  Assessment.  To  achieve  these  goals  it  is  irnportant 
that  the  pm^  managefhe  aware 

program  so  ftat  be  made 

TringX  p"uU  a.  the  L  of  a  test  phase)  and  compared  to  the  planned 
reliability  growth  values. 

1  423  Controlling  Reliability  Growth.  These  assessments  provide  visibility  of 

achieve:;t'mL"on^def.eienciSwhJl^^^^^^^^ 

requirements,  management  can  control  the  growth  process. 

TVip  various  technioues  associated  with  reliability 

growthlnt^age^hm^ 

«ay  plans  and  P-gress  h^db-lcs  for  revtew. 

WUhout  this  impleLntation,  reliabihty  growth  cannot  truly  be  managed. 
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The  planned  growth  curve  and  milestones  are  only  targets.  They  do  not  imply 
that  reliability  will  automatically  grow  to  these  values.  On  the  contrary,  these  values  will 
be  attained  only  with  the  incorporation  of  an  adequate  number  of  effective  design  fixes 
into  the  system.  This  requires  dedicated  management  attention  to  reliability  growth.  The 
methods  in  this  guide  are  for  the  purpose  of  assisting  management  in  making  timely  and 
appropriate  decisions  to  ensure  sufficient  support  of  the  reliability  engineenng  design 
effort  throughout  the  development  testing  program. 

High  level  management  of  reliability  growth  is  necessary  in  order  to  have  available  all 
the  options  for  difficult  program  decisions.  For  example,  high  level  decisions  in  the 
following  areas  may  be  necessary  in  order  to  ensure  that  reliability  goals  are  achieved: 

•  Revise  the  program  schedule. 

•  Increase  testing. 

•  Fund  additional  development  efforts. 

•  Add  or  reallocate  program  resources. 

•  Stop  the  program  until  interim  reliability  goals  have  been  demonstrated. 

Although  some  of  these  options  may  result  in  severe  program  delay  or  significant 
increase  in  development  costs,  they  may  have  to  be  exercised  in  order  to  field  equipment 
that  meets  user  needs  and  has  acceptable  total  life  cycle  costs. 

1.4.4  Basic  Reliability  Activities.  Reliability  growth  management  is  part  of  the 
system  engineering  process.  It  does  not  take  the  place  of  the  other  basic  reliability 
program  activities  such  as: 

•  Design  predictions 

•  Apportionment 

•  Failure  modes  and  effects  analysis 

•  Stress  analysis 

Instead,  reliability  growth  management  provides  a  means  of  viewing  all  the 
reliability  program  activities  in  an  integrated  manner. 


1.4.5  Reliability  Growth  Process. 

1. 4.5.1  Basic  Process.  Reliability  growth  is  the  result  of  an  iterative  design 
process.  As  the  design  matures,  it  is  investigated  to  identify  actual  or  potential  sources  of 
failures.  Further  design  effort  is  then  spent  on  these  problem  areas.  The  design  effort  can 
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be  applied  to  either  product  design  or  manufacturing  process  design.  The  iterative 
process  can  be  visualized  as  a  simple  feedback  loop  as  m  Figure  1 .  This  illustrates  that 
there  are  three  essential  elements  involved  in  achieving  reliability  growth: 

•  Detection  of  failure  sources, 


•  Feedback  of  problems  identified  and 


•  Redesign  effort  based  on  problems  identified. 

Identified  Problems 


L  (Re)  Design 

. 

Detection  of  _ 

Failure  Sources 

Figure  1.  Reliability  Growth  Feedback  Model. 

Furthermore,  if  failure  sources  are  detected  by  testing,  a  fourth  element  is  necessary; 
•  Fabrication  of  hardware. 

And,  following  redesign,  detection  of  failure  sources  serves  as. 

•  Verification  of  redesign  effort. 


Identified  Problems 


Figure  2.  Reliability  Growth  Feedback  Model  with  Hardware. 

1. 4.5.2  Growth  Rate.  The  rate  at  which  reliability  grows  is  dependent  on; 

•  how  rapidly  activities  in  this  loop  can  be  accomplished, 

•  how  significant  the  identified  problems  are,  and 
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•  how  well  the  redesign  effort  solves  the  identified  problems  without 
introducing  new  problems. 

Any  of  these  activities  may  act  as  a  bottleneck.  The  cause  and  degree  of  the 
bottleneck  may  vary  from  one  development  program  to  the  next,  and  even  within  a  single 
program  may  vary  from  one  stage  of  development  to  the  next. 

1.4.6  Reliability  Growth  Management  Control  Processes.  Figures  1, 2,  3,  and 
5  illustrate  the  growth  process  and  associated  management  processes  in  a  skeleton  form. 
This  type  of  illustration  is  used  so  that  the  universal  features  of  these  processes  may  be 
addressed.  The  representation  of  an  actual  program  or  program  phase  may  be 
considerably  more  detailed.  This  detailing  may  include  specific  inputs  to,  and  outputs 
from,  the  growth  process,  additional  activity  blocks,  and  more  explicit  decision  logic 
blocks. 


1. 4.6.1  Basic  Methods.  There  are  two  basic  ways  that  the  manager  evaluates  the 
reliability  growth  process.  The  first  method  is  to  utilize  assessments  (quantitative 
evaluations  of  the  current  reliability  status)  that  are  based  on  information  from  the 
detection  of  failure  sources.  The  second  method  is  to  monitor  the  various  activities  in  the 
process  to  assure  himself  that  the  activities  are  being  accomplished  in  a  timely  manner 
and  that  the  level  of  effort  and  quality  of  work  are  in  compliance  with  the  program  plan. 
Each  of  these  methods  complements  the  other  in  controlling  the  growth  process. 


Identified  Problems 


Figure  3.  Reliability  Growth  Management  Model  (Assessment). 
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1.4.6.2  -  Comparison  of  M®**^®*^*-  •  tTsupplement  the 

however,  the  monitonng  approach,  whic  ^  ^  program.  This  is  often 

SfJ'laS  oVs^fficL  objecfve  information  in  the  early  program 

stages. 

1  d  n  a  Assessment  Figure  3  illustrates  how  assessments  may  be  used  m 

eon  Jn:t  ^Itb  process.  is  a  more 

conventional  reliability  program  '”‘”‘*8™“' “  S”  i-ssmeL  are  compared.  Second, 
objectively  developed  growth  “  of 

the  assessment  methods  used  assessment  and  the 

the  present  equipment  configuration.  P  „_„gggjj^CT  as  planned,  better  than 

planned  value  will  suggest  ^het  er  ?  falling  g^ort,  new  strategies  should 

planned,  or  not  as  well  as  p  ■  ,  reassignment  of  resources  to  work  on 

be  developed.  These  strategies  m  y  ,  ■  ^  t  of  the  timeframe  or  a  re-examination 

reliability  growth  and  assessments. 


Reliability 


Planned  Growth  - 

Assessed  Growth  ■ 

, 

/  Test  Phase  1 

Test  Phase  2  j  Test  Phase  3 

Cumulative  Units  of  Test  Duration 


Figure  4.  Example  of  Planned  Growth  and  Assessments. 

14  6  4  Monitoring.  Figure  5  illustrates  control  of  the  growth  process  by 
.onitoHngthe  ^Jo^wth  ac/vities. 

performance  of  the  activities  involve  ,  ^  .  Nevertheless  this  activity  is  a 

definitive  than  management  based  on  asses  ments^  approach  to 

valuable  complement  to  relia  i  le^el  of  effort  and  quality  of  work 

reliability  growth  managem  .  Up^^vilv  on  the  technical  judgment  of  the 

rvrKrrgtreSfaium^^^^^^^ 

Tec^fgutrSTel  ca,c„.a.io„s.  One  of  .he  beher  examples 
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.  •  oM/  TViedesim  review  is  a  planned  monitoring 

of  a  monitoring  activity  is  the  performance  requirements  during 

of  a  product  design  to  assure  J  serve  to  determine  the  progress  bemg 

operLnal  use.  Su^  oW^  Sf  ^  the  «,ost  si^ficant  aspect  of  the 

Te"S  -tS  on  technical  jud^ent.  in  addition  to  quanhiattve 

assessments  of  progress. 


Identified 


(Re)  Design  / 
Prntotvnes  _ 


Detection  of 
Failure  _ 


;] 


Activities 

Monitoring 


Reliability 
Program  Plan 


Decision 


Figure  5.  ReliabUity  Growth  Management  Model  (Monitoring). 

,4.y  Factors InnneneingGr^thCurv^^^^ 

factors  that  affect  the  slwpe  of  the  ^o  ^  configuration  under  test,  the 

SSi^°SgSr:aL  tmits  of  measure  for  test  duration  all  mfluence 

the  growth  curve’s  shape. 

1.4.7.1  Stages  of  the  Development  for  each  stage, 

development  program  is  development  pmgram  need  not  be  the 

.  Proposal.  There  is  no  designs..  In  this 

":g°etllErove;  what  are  the  requir^ents,  can  they  be  met,  and  tf  so, 
how  and  nt  wh3-t  cstimntcd  cost. 

.  Conceptual.  Experimental  prototype  ar^mU  at 

little  resemblance  to  the  actual  system.  They  ate  tor  p 

u  thp  final  system  are  built  and  tested.  This 
•  ^.:g"XTrp“erfonn^^^  and  reliability  obiectives  for  the  system. 
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•  Full  Scale  Development.  Systems  built  as  though  they  were  in  production  are 
tested  to  work  out  final  design  details  and  manufacturing  procedures. 

Quantitative  reliability  growth  management  can  be  used  during  the  validation  and 
full-scale  development  stages  of  the  program.  It  could  be  argued  that  the  different  nature 
of  the  testing  going  on  in  these  stages  is  different  enough  to  cause  different  rates  of 
growth  to  occur.  How  much  different  the  types  of  testing  are  determines  how  they  will 
be  treated  in  creating  the  planning  growth  curve.  This  will  be  further  discussed  in 
Section  1.4.7. 6. 

1. 4.7.2  Test  Phases.  Within  a  development  stage  it  is  quite  likely  that  testing 
will  be  broken  up  into  alternating  time  periods  of  active  testing  followed  by  none.  Each 
period  of  active  testing  can  be  viewed  as  a  testing  phase.  Also,  within  a  development 
stage  it  is  quite  likely  that  more  than  one  type  of  testing  will  be  going  on  (e.g., 
performance  testing).  If  these  other  tests  that  are  not  specifically  for  reliability  follow  the 
intended  operating  environment  and  the  intended  use  stresses  well  enough,  and  if  design 
changes  are  made  on  the  basis  of  these  tests,  then  the  information  gathered  may  be 
incorporated  into  the  reliability  growth  test  data  base.  These  would  also  be  called 
reliability  growth  testing  phases.  It  is  to  be  expected  that  the  reliability  will  grow  firom 
one  phase  to  the  next.  The  reliability  growth  planning  curve  should  reflect  this. 

1. 4.7.3  System  Configurations.  In  an  absolute  sense,  any  change  to  the  design 
of  a  system  constitutes  a  new  configuration.  For  our  purposes,  we  will  term  a  specific 
design  a  new  configuration  if  there  has  been  one  significant  design  change,  or  enough 
little  design  changes,  that  cause  an  obviously  different  failure  rate  for  the  system.  It  is 
possible  that  two  or  more  testing  phases  could  be  grouped  together  for  analysis  based  on 
the  configuration  tested  in  these  phases  being  substantially  unchanged.  It  is  also  possible 
that  one  design  change  is  so  effective  at  increasing  reliability  that  a  new  configuration 
could  occur  within  a  test  phase.  System  configuration  decisions  can  also  be  made  on  the 
basis  of  engineering  judgement.  Obviously,  the  configuration  under  test  has  great 
influence  on  the  growth  curve. 

1.4.7.4  Timing  of  Fixes.  The  replacement  of  a  part  with  another  part  identical  to 
the  first  is  termed  a  repair.  Replacing,  or  eliminating,  a  part  due  to  a  design  change  is 
termed  a  fix.  Fixes  are  intended  to  reduce  the  rate  at  which  the  system  fails.  Repairs 
make  no  change  in  the  failure  rate  of  the  system.  The  time  of  insertion  of  a  fix  affects  the 
pattern  of  reliability  growth. 

1. 4.7.4, 1  Test-Fix-Test.  In  an  absolutely  pure  test-fix-test  program,  when  a 
failure  is  observed,  testing  stops  until  a  design  change  is  implemented  on  the  system 
under  test.  When  the  testing  resumes,  it  is  with  a  system  that  has  incrementally  better 
reliability.  The  graph  of  reliability  for  this  testing  strategy  is  a  series  of  small  increasing 
steps,  with  each  step  stretching  out  longer  to  represent  a  longer  time  between  failures. 
Such  a  graph  can  be  approximated  by  a  smooth  curve.  See  Figure  6. 
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Reliability 


Figure  6.  Graph  of  Reliability  in  a  Test-Fix-Test  Program. 

Such  a  pure  test-fix-test  program  is  impractical  in  most  situations.  Testing  is 
likely  to  continue  with  a  repair,  and  the  fix  will  be  implemented  later.  Nevertheless,  if 
fixes  are  inserted  as  soon  as  possible  and  while  testing  is  still  proceeding,  the  stair  step 
like  reliability  increases  and  the  shape  of  the  approximating  curve  will  be  similar,  but  rise 
at  a  slower  rate.  This  is  due  to  the  reliability  remaining  at  the  same  level  that  it  was  at 
when  the  failure  happened  until  the  fix  is  inserted.  Thus  the  steps  will  all  be  of  longer 
length,  but  the  same  height.  Continuing  to  test  after  the  fix  is  inserted  will  serve  to  verify 
the  goodness  of  the  design  change. 

1.4.7.4.2  Test-Find-Test  During  a  test-find-test  program  the  system  is  also 
tested  to  determine  problem  failure  modes.  However,  unlike  the  test-fix-test  program, 
fixes  are  not  incorporated  into  the  system  during  the  test.  Rather,  the  fixes  are  all  inserted 
into  the  system  at  the  end  of  the  test  phase  and  before  the  next  testing  period.  Since  a 
large  number  of  fixes  will  generally  be  incorporated  into  the  system  at  the  same  time, 
there  is  usually  a  significant  jump  in  system  reliability  at  the  end  of  the  test  phase.  The 
fixes  incorporated  into  the  system  between  test  phases  are  called  delayed  fixes.  See 
Figure  7. 


Jump  due  to 

r  • 

insertion  of 

/ 

Reliability 

delayed  fixes  ' 

( 

Measure  of  Test  Duration 

Figured  Graph  of  Reliability  in  a  Test-Find-Test  Program. 

1.4.7.4.3  Test-Fix-Test  with  Delayed  Fixes.  The  test  program  commonly  used 
in  development  testing  employs  a  combination  of  the  two  types  of  fix  insertions 
discussed  above.  In  this  case,  some  fixes  are  incorporated  into  the  system  during  the  test 
while  other  fixes  are  delayed  until  the  end  of  the  test  phase.  Consequently,  the  system 
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reliability  will  generally  be  seen  as  a  smooth  process  during  the  test  phase  and  then  jump 
due  to  the  insertion  of  the  delayed  fixes.  See  Figure  8. 


Reliability 


Figure  8.  Graph  of  Reliability  in  a  Test-Fix-Test  Program  with  Delayed  Fixes. 

14  7  5  Combined  Influences  of  Factors  on  Reliability  Growth  Curve  Shape. 
In  order  io'reach  the  goal  reliability,  the  development-testing  program  will  ^aUy 
of  several  major  test  phases.  Within  each  test  phase  the  fix  insertion  may  be  earned  out 
in  any  one  of  the  three  ways  discussed  above.  As  an  example,  suppose  that  tes  mg  were 
conducted  during  the  validation  and  full-scale  development  stages  oftheprop^^  Each 
stage  would  have  at  least  one  major  test  phase,  implying  a  m, mmum  ™jl” 

phies  for  the  program.  In  this  case,  there  would  be  3  -  9  general  ways  the  reliability 

may  grow  during  the  development  test.  See  Figure  9. 


9.1 

e 

Phase  1 1 

Phase  2 

9.4 

% 

Phase  11 

Phase  2 

9.7 

o 

Phase  1] 

Phase  2 

9.5 


Phase  1 1  Phase  2 


Figure  9.  The  Nine  Possible  General  Growth  Patterns  for  Two  Test  Phases. 
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Row  1  shows  Phase  1  as  having  all  fixes  delayed  until  the  end  of  the  testing  phase.  Row 
2  shows  Phase  1  as  having  some  fixes  inserted  during  test  and  some  delayed.  Row  3 
shows  Phase  1  as  having  all  fixes  inserted  during  test,  with  none  delayed.  Column  1 
shows  Phase  2  as  having  all  fixes  delayed  until  the  end  of  the  testing  phase.  Column  2 
shows  Phase  2  as  having  some  fixes  inserted  during  test  and  some  delayed.  Column  3 
shows  Phase  2  as  having  all  fixes  inserted  during  test,  with  none  delayed. 

Figures  9.1  and  9.9  represent  the  two  extremes  in  possible  growth  test  patterns. 
There  are  some  distinct  statistical  advantages  to  following  a  complete  test-fix-test 
program: 

•  The  estimated  value  of  reliability  at  any  point  along  the  smooth  growth  curve 
is  an  instantaneous  value.  That  is,  it  is  not  dragged  down  by  averaging  with 
the  failures  that  accrued  due  to  earlier  (and  hopefully)  less  reliable 
configurations. 

•  Confidence  limits  about  the  true  value  of  reliability  can  be  established. 

•  While  the  impact  of  the  jumps  in  reliability  can  be  assessed  using  a  mix  of 
some  engineering  judgement  (this  will  be  discussed  in  the  section  on 
Reliability  Growth  Projection)  and  direct  calculation,  the  estimate  of 
reliability  in  a  test-fix-test  program  is  based  solely  on  data. 

•  In  a  test-fix-test  program,  the  goodness  of  the  design  changes  is  continuously 
being  assessed  in  the  estimate  of  reliability. 

A  development  stage  may  consist  of  more  than  one  distinct  test  phase.  For 
example,  suppose  that  testing  is  stopped  part  way  through  the  full-scale  development 
stage,  and  delayed  fixes  are  incorporated  into  the  system.  The  testing  in  this  case  may  be 
considered  as  two  major  test  phases  during  this  stage,  giving  three  phases  for  the  whole 
program.  If  a  program  had  three  major  test  phases  then  there  would  be  3  =27  patterns 
of  reliability  growth.  Obviously  this  manner  of  determining  the  possible  number  of 
growth  patterns  can  be  extended  to  any  number  of  phases. 

1.4.7.6  Growth  Curve  Reinitialization.  The  differences  in  the  growth  curves 
between  phases  shown  in  Figures  9.5  and  9.6  represent  the  difference  mentioned  in  the 
last  paragraph  of  Section  1. 4.7.1.  Underlying  Figure  9.6  is  the  assumption  that  the  testing 
environment  and  engineering  efforts  are  the  same  across  test  phases,  thus  the 
continuation  of  the  same  growth  curve  into  the  succeeding  phase,  after  the  jump  for 
delayed  fixes.  In  Figure  9.5  some  factor  influencing  the  rate  of  growth  has  substantially 
changed  between  the  phases,  which  is  reflected  in  a  new  growth  curve  for  the  succeeding 
phase.  This  is  called  reinitializing  the  growth  curve.  It  must  be  emphasized  that 
reinitialization  of  a  growth  curve  is  only  justified  if  the  testing  environment  is  so  different 
as  to  introduce  a  new  set  of  failure  modes,  or  the  engineering  effort  is  so  different  as  to  be 
best  represented  as  a  totally  new  program. 
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1.4.7.7  Shape  Changes  Due  to  Calendar  Time.  Reliability  growth  is  often 
depicted  as  a  fimction  of  test  time  for  evaluation  purposes.  For  management  and 
presentation  purposes  it  may  be  desirable  to  portray  reliability  growth  as  » Junction  of 
calendar  time.  This  can  be  accomplished  by  determining  the  number  Junits  of  test 
duration  that  will  have  been  completed  at  each  measure  point  in  calendar  time  and  then 
plotting  the  value  that  corresponds  to  the  completed  test  duration  above  that  calendar 
point.  This  is  a  direct  fimction  of  the  program  schedule.  Figure  10  shows  the  reliability 
growth  of  a  system  as  a  fimction  of  test  time  and  calendar  time. 


Reliability 


Figure  10.  Comparison  of  Growth  Curves  Based  on  Test  Duration  Vs  Calendar 

Time. 

1.4.8  Reliability  Growth  Concepts. 

1  4  8.1  Levels  of  Consideration  for  Growth.  Planning  and  controlling 
reliability  ^owth  can  be  divided  as  to  levels  of  consideration  along  both  a  program  basis 
and  an  item  under  test  basis. 

•  Program  considerations: 

•  Global:  This  approach  treats  reliability  growth  on  a  total  basis  over  the 
entire  development  program. 

•  Local:  The  other  approach  treats  reliability  growth  on  a  phase-by-phase 
basis. 

•  Item  Under  Test  considerations: 

•  System  Level:  The  entire  system  as  it  is  intended  to  be  fielded  is  tested. 
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•  Subsystem  Level:  The  obvious  meaning  is  the  testing  of  a  major  and 
reasonably  complex  component  of  the  whole  system  (e.g.,  an  engine  for  a 
vehicle).  Sometimes,  the  subsystem  would  seem  to  be  an  autonomous 
unit,  but  because  the  requirement  is  for  this  unit  to  operate  in  conjunction 
with  other  units  to  achieve  an  overall  functional  goal  it  is  really  only  part 
of  “the  system”  (e.g.,  radar  for  an  air  defense  system). 

The  appropriate  level  of  consideration  can  be  different  at  different  times  within 
the  development. 

1  4.8.2  Analysis  of  Previous  Programs.  Analysis  of  previous  similar  programs 
is  used  to  develop  guidelines  for  predicting  the  growth  during  future  programs  Such 
analysis  may  be  performed  on  either  overall  programs  or  individual  progr^  phases,  or 
both  Of  particular  interest  are  the  patterns  of  growth  observed  and  the  effect  of  program 
characteristics  on  initial  values  and  growth  rates.  The  U.S.  Army  Matenel  Systems 
Analysis  Activity  (AMSAA)  has  conducted  a  data  study,  [2],  that  is  a  useful  guide  in 
choosing  appropriate  growth  rates  for  various  system  types. 

1.4.9  Planning. 

1.4.9.1  Planned  Growth  Curve.  The  planned  growth  curve  is  a  picture  of  the 
anticipated  reliability  growth  for  the  entire  program.  It  is  an  essential  part  of  the 
reliability  growth  management  methodology  and  is  important  to  any  reliability  program. 
This  curve  is  constructed  early  in  the  development  program  generally  before  hard 
reliability  data  are  obtained  and  is  typically  a  joint  effort  between  the  program  manager 
and  contractor.  Its  primary  purpose  is  to  provide  management  with  guidelines  as  to  what 
reliability  can  be  expected  at  any  stage  of  the  program  and  to  provide  a  basis  for 
evaluating  the  actual  progress  of  the  reliability  program  based  upon  generated  reliabi  i  y 
The  planned  growth  curve  can  be  constructed  on  a  phase-by-phase  basis,  bee 

Figure  11. 


Determination  of  pattern  and  phase  characteristics  that  influence  growth  curves. 


Figure  11.  Development  of  Planned  Growth  Curve  on  a  Phase  by  Phase  Basis. 
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14  9.2  Idealized  Growth  Curve.  An  Idealized  Growth  Curve  is  a  plarmed 
growth  c'un-e  that  consists  of  a  single  smooth  curve  based  “ 

12. 


Specific  Idealized  Growth  Curve 


/ 

I 


Appropriate  for  this  development  program 

Figure  12.  Global  Analysis  Determination  of  Planned  Growth  Curve. 


1.4.10  Tracking. 


1  4  10  1  Demonstrated  Reliability.  A  demonstrated  reliability  value  is  based  on 
actual  t«^dl  a^dTa^  estimate  of  the  current  attained  reliability.  The  assessment  is 
made  on  the  system  configuration  currently  undergoing  test, 

g^^h  established  over  the  history,  to  date,  of  the  development  program. 


1  4  10.2  Reliability  Growth  Tracking  Curve. 

curve  is  the  curve  that  best  fits  the  data  being  analyzed. 


The  reliability  growth  tracking 
It  can  be  based  on  data  solely 
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within  one  phase  or  data  from  several  phases.  Whatever  period  of  testing  is  used  to  form 
a  database,  this  curve  is  the  statistical  best  representation  from  a  family  of  growth  curves 
of  the  overall  reliability  growth  of  the  system.  It  depicts  the  trend  of  growth  that  has  been 
established  over  the  database.  Thus,  if  the  database  covers  the  entire  program  to  date,  the 
right  end  point  of  this  curve  is  the  current  demonstrated  reliability.  See  Figure  13. 


Reliability 


Figure  13.  Reliability  Growth  Tracking  Curve. 

1.4.11  Projection. 

1.4.11.1  Extrapolated  Reliability.  Extrapolating  a  growth  curve  beyond  the 
currently  available  data  shows  what  reliability  a  program  can  be  expected  to  achieve,  as  a 
function  of  additional  test  duration,  provided  the  conditions  of  test  and  the  engineering 
effort  to  improve  reliability  are  maintained  at  their  present  levels  (i.e.,  the  established 
trend  continues). 

1.4.11.2  Projected  Reliability.  A  reliability  projection  is  an  assessment  of 
reliability  that  can  be  anticipated  at  some  future  point  in  the  development  program.  The 
projection  is  based  on  the  achievement  to  date  and  engineering  assessments  of  future 
program  characteristics.  Projection  is  a  particularly  valuable  analysis  tool  when  a 
program  is  experiencing  difficulties  because  it  enables  investigation  of  program 
alternatives. 


Reliability 


Extrapolation 


Figure  14.  Extrapolated  and  Projected  Reliabilities. 
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2.  RELIABILITY  GROWTH  PLANNING 

2.1  System  Level  Planning. 

2.1.1  Introduction.  The  material  in  Section  2.1  is  from  Reference  [1]. 

A  well  thought  out  reliability  growth  plan  can  serve  as  a  significant  management  tool  in 
scoping  out  the  required  resources  to  enhance  system  reliability  and  demonstrate  the  system 
reliability  requirement.  The  principal  goal  of  the  growth  test  is  to  enhance  reliability  by  the 
iterative  process  of  surfacing  failure  modes,  analyzing  them,  implementing  corrective  actions 
(fixes),  and  testing  the  "improved"  configuration  to  verify  fixes  and  continue  the  growth  process 
by  surfacing  remaining  failure  modes.  If  the  growth  test  environment  during  engineering  and 
manufacturing  development  (EMD)  reasonably  simulates  the  mission  environment  stresses  then 
it  may  be  feasible  to  use  the  growth  test  data  to  statistically  demonstrate  the  technical,  i.e., 
engineering,  requirement  (denoted  by  TR)  for  system  reliability.  Such  use  of  the  growth  test 
data  could  eliminate  the  need  to  conduct  a  follow-on  reliability  demonstration  test.  The  classical 
demonstration  test  requires  that  the  system  configuration  be  held  constant  throughout  the  test. 
This  type  of  test  is  principally  conducted  to  assess  and  demonstrate  the  reliability  of  the 
configuration  under  test. 

Associated  with  the  demonstration  test  are  statistical  consumer  and  producer  risks.  In  our 
context,  they  are  frequently  termed  the  Government  and  contractor  risks,  respectively.  In  broad 
terms,  the  Government  risk  is  the  probability  of  accepting  a  system  when  the  true  technical 
reliability  is  below  the  TR  and  the  contractor  risk  is  the  probability  of  rejecting  a  system  when 
the  true  technical  reliability  is  at  least  the  contractor's  target  value  (set  above  the  TR).  An 
extensive  amount  of  test  time  may  be  required  for  the  reliability  demonstration  test  to  suitably 
limit  these  statistical  risks.  Moreover,  this  allotted  test  time  would  be  principally  devoted  to 
demonstrating  the  system  TR  associated  with  the  configuration  under  test  instead  of  to  enhancing 
the  system  reliability  . through  the  reliability  growth  process  qf  sequential  configuration 
improvement.  In  today's  austere  budgetary  environment,  it  is  especially  important  to  make 
maximum  use  of  test  resources.  With  proper  planning,  a  reliability  growth  program  can  be  an 
efficient  procedure  for  demonstrating  the  system  reliability  requirement  while  reliability 
improvements  are  being  achieved  via  the  growth  process. 

2.1.2  Background.  During  a  reliability  growth  test  phase,  the  system  configuration  is 
changing  due  to  the  activity  of  surfacing  failure  modes,  analyzing  the  modes,  and  implementing 
fixes  to  the  surfaced  modes.  It  is  often  reasonable  to  portray  this  reliability  growth  in  an 
idealized  manner,  i.e.,  by  a  smooth  rising  curve  that  captures  the  overall  pattern  of  growth.  The 
curve  relates  a  measure  of  system  reliability,  e.g.,  mean-time-between-failures  (MTBF),  to  test 
duration  (e.g.,  hours).  The  functional  form  used  to  express  this  relationship  in  MIL-HDBK-189 
[2]  is  given  by 
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M[t)  =  {M,l(\-a))  {tit,) 


(1) 


In  this  equation,  M(t)  typically  denotes  the  MTBF  achieved  after  t  test  hours.  The  exponent  a  is 
termed  the  growth  rate  and  represents  the  slope  of  the  assumed  linear  relationship  between 
ln{M(t)}  and  ln(t),  where  In  denotes  the  base  e  logarithm  function.  The  parameters  t,,  M ,  may 
be  thought  of  as  defining  the  initial  conditions.  In  particular,  M ,  may  be  interpreted  as  the 
MTBF  associated  with  the  initial  configuration  entering  the  reliability  growth  test.  In  this 
interpretation,  t,  would  be  the  planned  cumulative  test  time  until  one  or  more  fixes  are 
incorporated.  An  alternate  and  more  general  interpretation  of  M ,  and  t,  would  be  to  regard 
M ,  as  the  anticipated  average  MTBF  over  an  initial  test  period  t , . 

In  the  above  discussion,  we  have  referred  to  M(t)  as  the  MTBF  and  have  measured  test 
duration  by  time  units,  e.g.,  t  hours.  We  will  continue  to  refer  to  M(t)  and  test  duration  t  in  this 
fashion;  however,  more  generally,  M(t)  may  denote  mean-miles-to-failure  or  mean-rounds-to- 
failure  (for  a  large  number  of  rounds).  The  corresponding  measures  of  test  duration  would  be 
test  mileage  or  rounds  expended,  respectively. 

As  indicated  in  Section  2.1.1,  we  shall  consider  using  the  data  generated  during  the 
reliability  growth  test  phase  to  demonstrate  the  system  reliability  technical  requirement  (TR)  at  a 
specified  confidence  level  y.  This  section  addresses  the  case  where  the  data  consists  of 
individual  failure  times  0</,  <t^<  ...  <i„  <T  for  n  observed  mission  reliability  failures  during 

test  time  T,  where  Equation  ( 1 )  is  assumed  to  hold  for  0<t  <  T .  Since  the  MIL-HDBK- 189 
growth  model  governed  by  Equation  (1)  is  being  assumed  in  this  section,  we  shall  also  require 
that  the  observed  number  of  failures  by  test  duration  t,  denoted  by  N(t),  be  a  non-homogeneous 

Poisson  process  with  intensity  function  f{t)  =  { . 

The  growth  curve  planning  parameters  a,  t , ,  M , ,  and  the  test  time  T  should  be  chosen  to 
reasonably  limit  the  consumer  (Government)  and  producer  (contractor)  statistical  risks  referred 
to  in  Section  2.1.1.  Prior  to  presenting  the  relationship  between  these  risks  and  the  parameters 
mentioned  above,  it  is  instructive  to  review  the  determination  of  these  risks  for  a  reliability 
demonstration  test  based  on  a  constant  configuration. 

The  parameters  defining  the  reliability  demonstration  test  consist  of  the  test  duration  , 
and  the  allowable  number  of  failures  c.  Define  the  random  variable  to  be  the  number  of 
failures  that  occur  during  the  test  time  .  Denote  the  observed  value  of  F^^^  by  .  Then 
the  "acceptance"  or  "passing"  criterion  is  simply  <  c . 

Let  M  denote  the  MTBF  associated  with  the  constant  configuration  under  test.  Then  F^^^ 
has  the  Poisson  probability  distribution  given  by 


Prob(F,,  =  i) 


g-To^/M 


i! 
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Thus  the  prohahility  of  acceptance,  denoted  by  Prob(A;  M,  c,  )»  ^  function  of  M,  c,  and 

T^dem  is  given  by 

c 

Prob(A;M,c,Ti,,,)  =  Prob(F,<c)  =  XProb(F,=i) 


c 

=  S 


-T^IU 


i=0 


i! 


(3) 


To  ensure  "passing  the  demonstration  test"  is  equivalent  to  demonstrating  the  TR  at  confidence 
level  y  (e.g.,  y  =  0.80  or  y  =  0.90),  we  must  choose  c  such  that 

/^<c  o  TRit^U)  W 

where  TR>0  and  ^^(4*,)  denotes  the  value  of  the  100  y  percent  lower  confidence  bound  when 

failures  occur  in  the  demonstration  test  of  length  .  Note  that  is  a  lower 

confidence  bound  on  the  true  (but  unknown)  MTBF  of  the  configuration  under  test.  It  is  well 
known  (see  Proposition  1  in  Appendix  C)  that  the  following  choice  of  c  satisfies  (4): 

Choose  c  to  be  the  largest  non-negative  integer  k  that  satisfies  the  inequality 


Z 


e 


•Toan'TR 


(T^/TR)‘ 

i! 


Note  c  is  well-defined  provided 


<  l-y 


exp(-TD,„/TR)  <  \-r 


(5) 

(6) 


Throughout  this  section  we  shall  assume  (6)  holds  and  that  c  is  defined  as  above. 

Recall  that  the  operating  characteristic  (OC)  curve  associated  with  a  reliability 
demonstration  test  is  the  graph  of  the  probability  of  acceptance,  i.e.,  Prob  (A;M,c,  )  given  in 

Equation  (3),  as  a  function  of  the  true  but  unknown  constant  MTBF  M  as  depicted  on  Figure  1 . 
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Figure  1.  Example  OC  Curve  for  Reliability  Demonstration  Test. 

The  Government  (or  consumer  risk)  associated  with  this  curve,  called  the  Type  II  nsk,  is  defined 
by 

Type  II  A  Prob  ( A;  TR,  c,  T^ )  (7) 

Thus,  by  the  choice  of  c, 

Type  II  <  1-r 

For  the  contractor  (producer)  to  have  a  reasonable  chance  of  demonstrating  the  TR  with 
confidence  y,  the  system  configuration  entering  the  reliability  demonstration  test  must  often  have 
a  MTBF  value,  say  (the  contractor's  goal  MTBF)  that  is  considerably  higher  than  the  TR. 
The  probability  that  the  producer  fails  the  demonstration  test  given  the  system  under  test  has  a 
true  MTBF  value  of  Mq  is  termed  the  producer  (contractor)  or  Type  I  risk.  Thus 

Type  I  =  1  -  Prob(A;M<.,c,T^)  (9) 

If  the  Type  I  risk  is  higher  than  desired,  then  either  a  higher  value  of  Mq  should  be  attained 
prior  to  entering  the  reliability  demonstration  test  or  should  be  increased.  If  T^em  is 
increased  then  c  may  have  to  be  readjusted  for  the  new  value  of  to  remain  the  largest  non¬ 
negative  integer  that  satisfies  inequality  (5). 
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The  above  numbered  equations  and  inequalities  express  the 
reliability  demonstration  test  parameters  c,  ,  the  requirement  parameters  TO.  y,  and  the 

totomtlifnSS^eoff  rJJs'^ftavol^^^^  Sse’parametem  for  pla^ng  reliability 

the  next  section  we  shall  present  relationships  between  the  definmg 
rara^'tm  for  areliahility  grow* curve  (M, .  r, .  a,  and  T).  the  requnement ParantJ-  (™ 
and  y)  and  the  associated  statistical  risk  parameters  (the  consumer  and 

thesi  relationsUps  are  in  hand,  tradeoffs  between  these  parameters  may  be  uhhz^  to  consider 
.1.n,nuetrari.,f  the  TR  at  confidence  level  y  by  utilizmg  rehabihty  growth  test  data. 

2  1  3  Reliability  Growth  Operating  Characteristic  (OC)  Analysis.  In  Previous 
section*  it  was  noted  Sit  for  a  reliability  demonstration  test,  passing  the  test  could  be  stated  in 
SSwab ‘dumber  of  failures,  c.  I.  was  noted  that  if  c  is  properly  chosen,  then 

passing  the  test  is  equivalent  to  demonstrating  the  TO  at  confidence  level  y,  t.e.. 


fa 

obs 


TR<^r(f,J 


Sn  fSmi  must  be  considered  when  using  reliability  growth  test  data  to  demonstmte  *e 
Tr*  a  specified  confidence  level  y.  Thus,  the  "acceptance"  or  "passing"  cntenon  must  be  sta 
dLtlv  in  terms  of  the  y  lower  confidence  bound  on  M(T)  calculated  from  the  rehabihty  growth 

data.  These  data  will  he  denoted  by  (m  s)  ,n 

growth  test  of  duration  T  and  s  =  ( t, ,  ,  •  •  • ,  )  is  the  vector  oi  cuuiui 

particular,  r,  denotes  the  cumulative  test  time  to  the  i"  failure  and  0<r.  <t  <t  <T  iox 

a.  >  1  We  shall  also  refer  to  the  random  vector  (N,  S)  which  takes  on  values  (n,  s)  for  «  >  1 . 
Me'ss  S  lughout  the  remainder  of  Section  2  (N,  S)  will  he  condittoned  on 

A^>1. 

Using  the  lower  confidence  bound  methodology  developed  for  reliability  growth  data  by 
Crow  in  [3],  we  shall  define  our  acceptance  criterion  by  the  inequa  ity 


TR  <  ly  (n,  s) 


(10) 


where  ^  .(n,^)  is  the  y  statistical  lower  confidence  bound  on  M(T),  calculated  as  in  [3]  for  n  >  1 . 
Thus,  the  probability  of  acceptance  is  given  by 


Prob  (TR  <  Lt  (N,S)  ) 


(11) 


where  the  random  variable  i,(W.S)  takes  on  the  value  <,(n,s)  when  (N,  S)  takes  on  the  value 
(n,  s). 

In  accordance  with  [3],  for  n  ^  1 ,  we  define 
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(n,s)  A 


^  2n 


M„(T) 


(12) 


where  z^(n)  is  the  unique  positive  value  of  z  such  that 


(13) 


In  the  above,  the  function  /,  denotes  the  modified  Bessel  function  of  order  one  defined  as 
follows: 


I.(z)  A 

■  “  trjKj-i)! 


(14) 


In  Equation  (12),  M„{t)  denotes  the  maximum  likelihood  estimate  (mle)  for  M(T)  given 
in  MIL-HDBK-189  when  n  failures  are  observed.  As  discussed  in  MIL-HDBK-189, 


M,{T)  =  r/(nA) 


(15) 


where 


A  =  n/  Xln(T/L) 


i=I 


(16) 


The  distribution  of  (N,  S)  and  hence  that  of  Ly  (N,  S)  is  completely  determined  by  the  test 
duration  T  together  with  any  set  of  parameters  that  define  a  unique  reliability  growth  curve  of  the 
form  given  by  Equation  (1)  in  Section  2.1.2.  Thus,  the  value  of  a  probability  expression  such  as 
given  in  (1 1)  also  depends  on  T  and  the  assumed  underlying  growth  curve  parameters.  One  such 
set  of  parameters,  as  seen  directly  from  Equation  (1),  is  t,,  M , ,  a  together  with  T.  In  this 
growth  curve  representation,  t,  may  be  arbitrarily  chosen  subject  to  0<t,  <T.  Alternately,  scale 
parameter  X>0  and  growth  rate  a,  together  with  T,  can  be  used  to  define  the  growth  curve  by  the 
equation 

M(t)  =  0<t<T  (17) 


where  /3  =  \- a. 


Note  by  Equation  (17), 
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MX  =  (M(T));^T^-’ 

Thus,  the  growth  curve  can  also  be  expressed  as 

M(t)  =  (M(T))(t/Tr,  0<t<T  (19) 

By  Equation  (19)  we  see  that  the  distribution  of  (N,  S)  and  hence  that  of  L,  (N,  S)  is  determined 
by(a,T,M(T)). 

Unless  otherwise  stated,  throughout  the  remainder  of  this  section,  th®  distributions  for  (N, 
S)  and  for  random  variables  defined  in  terms  of  (N,  S)  will  be  with  respect  to  a  fixed  but 
unsnecified  set  of  values  for  a,  T,  M(T)  subject  only  to  a<l,  T>0,  and  M(T)>0.  Th®  sme 
considerations  apply  to  any  associated  probability  expressions  In  particular,  the  probability  o 

acceptance,  i.e.,  Prob  (TR<L,(N,  S)),  is  a  fimction  of  (a,  T,  M(T)). 

To  further  consider  the  probability  of  acceptance,  we  must  first  consider  several  properties 
of  the  system  of  lower  confidence  bounds  generated  by  Lr  (N,  S)  as  specified  via  Equations  (  ) 

thrmigh  (16)  The  statistical  properties  of  this  system  of  bounds  directly  follow  from  the 
Xe?ies  of  a  set  of  conditional  bounds  derived  by  Crow  in  [3],  These  latter  bounds  are 
conditioned  on  a  sufficient  statistic  W  that  takes  on  the  value 

w  =  (20) 


when  (N,  S)  takes  on  the  value  (n,  s). 

Let  L,  (N,  S;  w)  denote  the  random  variable  L,  (N,  S)  conditioned  on  W  =  to  [3] 
Crow  shows  that  4  (N,  S;  w)  generates  a  system  of  y  lower  confidence  bounds  on  (  ),  • 

Prob(L,(N,S;w)SM(T))  >  r  (2') 

for  each  set  of  values  (a.  T,  M(T))  subject  to  a<l ,  T>0.  and  Mm>0  Note  tot  the  value  of  w  is 
not  known  prior  to  conducting  the  reliability  growth  test.  Thus,  to  “tolate  an  curve  tot  test 
planning  i.e.,  a  priori,  we  wish  to  base  our  acceptance  critenon  on  L,  (N,  S)  as  m  (1 1 )  and  not  on 
SlronlionLl  Ldont  variable  L,  (N,  S;  w).  We  can  utilize  E^ttation  (21)  to  stow  ^aee  ^ 
Propositions  2,  3,  and  4  in  Appendix  C)  that  the  Type  II  or  consumer  nsk  fo  (  ) 
most  1-y  (for  any  a<l  and  T>0),  analogous  to  the  case  in  Section  2.1.2,  i.e., 

Typell  =  Prob(TR<Lr(N,S))  <  1-/  (22) 

for  any  a<l  and  T>0,  provided  M(T)  =  TR. 
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To  emphasise  the  functional  dependence  of  the  probability 
true  growth  curve  parameters  (a,  T,  M(T)),  we  shall  denote  this  probability  by  Prob  (A,  a,  , 

M(T)).  Thus, 

Prob(A;^jr,T,M(T))  A  Prob(TR  <  Lr(N,S))  (23) 

where  the  distribution  of  (N,  S)  and  hence  that  of  L,  (N,  S)  is  >lete™ned  by  W^can 
U  i,r.«,n  thft  Prob  (A-  a  T  M(T))  only  depends  on  the  values  of  M(T)/TR  or  equivalently 
Mf^^^Ttoo^td  EOT.  m  rati  M(T)/TR  is  analogous  to  fte 

consJant  configuration  reliabihty  demonsnahon  test  oM^^ 

6  in  Appendix  C): 


E(N)  =  T/{(1-«)M(T)} 


(24) 


and 


Prob  (A;a,  T,  M(T)  )  = 


(1-e 


)•*  Z 


n-1 


Prob 


X2n 


zl  (n)  2pd 


(25) 


where  p  A  E(N)  and  d  A  M(T)/TR. 

Note  (25)  shows  that  the  probability  of  acceptance  only  d^P^ids  on  p  and  d.  Thus,  we  shall 
subsequently  denote  the  probability  of  acceptance  by  Prob  (A;p,d). 


By  (22), 


Type  II  =  Prob(A;//,  1) 


< 


(26) 


Thus,  the  actual  value  of  die  Government  or  _r  d^ends  -  ^ 

"Jo "  Thr^:i::2:il  a "alu^th;  conlcmr  feels  he  can  achieve  for  the 
It  M  dfnote  the  contractof  s  MTBF  goal.  This  is  the  MTBF  value  the  contractor 

TR  at  confidence  level  y  (utilizing  the  generated  reliability  growth  test  data)  is  given  by 


25 


Type  1  =  1-  Prob  ( A; ) 


(27) 


where 


d,  =M,/TR 


and  //o  =  T/{(l-aj,)Mj.} 


(28) 


If  the  Type  I  risk  is  higher  than  desired,  there  are  several  ways  to  consider  reducing  this  nsk 
while  maintaining  the  Type  II  risk  at  or  below  l-y.  Since  Prob  (A;  d^)  is  zn  increasing 

function  of  Mg  and  4 ,  the  Type  I  risk  can  be  reduced  by  increasing  one  or  both  of  these 
quantities,  e.g.,  by  increasing  T. 

To  further  consider  how  the  Type  I  statistical  risk  can  be  influenced,  we  shall  express 
and  Mg  in  terms  of  TR,  T,  ,  and  the  initial  conditions  {M, ,  t,  )•  Using  Equations  (1)  and  (19) 

with  a  =  and  M(T)  =  ,  by  (28)  we  can  show 


M. /TR  =  d( 


M. 


(l-^rJtf^TR 


(29) 


and 


E(N)  =  Ac  =  (t/‘^/M,)T 


X-Uq 


(30) 


Note  for  a  given  requirement  TR,  initial  conditions  {M„t, ),  and  an  assumed  positive  growth 
rate  a.  the  contractor  risk  is  a  decreasing  function  of  T  via  Equations  (27),  (29),  and  (30). 
These  equations  can  be  used  to  solve  for  a  test  time  T  such  that  the  contractor  risk  is  a  speafied 
value.  The  corresponding  Government  risk  will  be  at  most  l-y  and  is  given  by  Equation  (26). 

Section  2.1.4  contains  two  examples  of  an  OC  analysis  for  planmng  a  reliability  growth 
program  The  first  example  illustrates  the  construction  of  an  OC  curve  for  given  initial 
Lditions  {M, ,  /;)  and  requirement  TR.  The  second  example  illustrates  the  iterative  solution 

for  the  amount  of  test  time  T  necessary  to  achieve  a  specified  contractor 

initial  conditions  ( M, ,  f;)  and  requirement  TR.  These  examples  use  Equations  (  ) 

rewritten  as  in  Equations  (1)  and  (24),  respectively,  i.e.. 


M(T)  = 


(Ml.] 

f  T^ 

^l-a  J 

lt,> 

and  E(  N )  = 


(l-<3r)M(T) 


(31) 


■he  quantities  d=  M(T)/TR  and  n  =  E(N)  are  then  used  to  obtain  an  approximation  to  Prob 
A;p,d).  Approximate  values  are  provided  in  Appendix  B  for  a  range  of  values  for  p  and  d.  The 
lature  of  this  approximation  is  also  discussed  in  Appendix  B. 
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2.1.4  Application. 

2.1.4.1  Example  1.  Suppose  we  have  a  system  under  development  that  has  a  technical 
requirement  (TR)  MTBF  of  100  hours  to  be  demonstrated  with  80  percent  confidence.  For  the 
developmental  program,  a  total  of  2800  hours  test  time  (T)  at  the  system  level  has  been 
predetermined  for  reliability  growth  purposes.  Based  on  historical  data  for  similar  type  systems 
and  on  lower  level  testing  for  the  system  under  development,  the  initial  MTBF  {M,)  averaged 
over  the  first  500  hours  {t, )  of  system-level  testing  was  expected  to  be  68  hours.  Using  these 
data,  an  idealized  reliability  growth  curve  was  constructed  such  that  if  the  tracking  curve 
followed  along  the  idealized  growth  curve,  the  TR  MTBF  of  100  hours  would  be  demonstrated 
with  80  percent  confidence.  The  growth  rate  (a)  and  the  final  MTBF  (M(T))  for  the  idealized 
growth  curve  were  0.23  and  130  hours,  respectively.  The  idealized  growth  curve  for  this 
program  is  depicted  on  Figure  2. 


Figure  2.  Idealized  Reliability  Growth  Curve. 

For  this  example,  suppose  we  want  to  determine  the  operating  characteristic  (OC)  curve 
for  the  program.  For  this,  we  need  to  consider  alternate  idealized  growth  curves  where  the  M(T) 
vary  but  the  M,  and  t,  remain  the  same  values  as  those  for  the  program  idealized  growth  curve; 
i.e.,  M !  =  68  hours  and  tj  =  500  hours.  In  varying  the  M(T),  this  is  analogous  to  considering 
alternate  values  of  the  true  MTBF  for  a  reliability  demonstration  test  of  a  fixed  configuration 
system.  For  this  program,  one  alternate  idealized  growth  curve  was  determined  where  M(T) 
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equals  the  TR  whereas  the  remaining  alternate  idealized  growth  curves  were  determined  for 
different  values  of  the  growth  rate.  These  alternate  idealized  growth  curves  along  with  the 
program  idealized  growth  curve  are  depicted  on  Figure  3. 


0  600  1000  1600  2000  2600  3000  3600 

Test  Time  ( t  Hours  ) 


Figure  3.  Program  and  Alternate  Idealized  Growth  Curves. 

Now,  for  each  idealized  growth  curve  we  find  M(T)  and  the  expected  number  of  failures 
E(N)  from  equation  (31).  Using  the  ratio  M(T)/TR  and  E(N)  as  entries  in  the  tables  contained  in 
Appendix  B,  we  determine,  by  double  linear  interpolation,  the  probability  of  demonstrating  the 
TR  with  80  percent  confidence.  This  probability  is  actually  the  probability  that  the  80  percent 
lower  confidence  bound  (80  percent  LCB)  for  M(T)  will  be  greater  than  or  equal  to  the  TR. 
These  probabilities  represent  the  probability  of  acceptance  (P(A))  points  on  the  OC  curve  for  this 
program  which  is  depicted  on  Figure  4.  The  M(T),  a,  E(N),  and  P(A)  for  these  idealized  growth 
curves  are  summarized  in  the  following  table: 


M(T) 

a 

E(N) 

P(A) 

100 

0.14 

32.6 

29.2 

0.23 

28.0 

0.48 

139 

0.25 

26.9 

0.58 

163 

0.30 

24.5 

191 

0.35 

22.6 

226 

0.40 

20.6 

28 


True  M(T) 


Figure  4.  Operating  Characteristic  (OC)  Curve. 

From  the  OC  curve,  the  Type  I  or  producer  risk  is  0.52  (1-0.48)  which  is  based  on  the 
program  idealized  growth  curve  where  M(T)  =  130.  Note  that  if  the  true  growth  curve  were  the 
program  idealized  growth  curve,  there  is  still  a  0.52  probability  of  not  demonstrating  the  TR  with 
80  percent  confidence.  This  occurs  even  though  the  true  reliability  would  grow  to  M(T)  -  130 
which  is  considerably  higher  than  the  TR  value  of  100.  The  Type  II  or  consumer  nsk,  which  is 
based  on  the  alternate  idealized  growth  curve  where  M(T)  =  TR  -  100,  is  0.15.  As  indicated  on 
the  OC  curve,  it  should  be  noted  that  for  this  developmental  program  to  have  a  producer  nsk  ot 
0.20,  the  contractor  would  have  to  plan  on  an  idealized  growth  curve  with  M(T)  -  167. 

2.1.4.2  Example  2.  Consider  a  system  under  development  that  has  a  technical 
requirement  (TR)  MTBF  of  1 00  hours  to  be  demonstrated  with  80  percent  confidence,  as  in 
Example  1.  The  initial  MTBF  (M, )  over  the  first  500  hours  (t; )  of  system  level  testing  for  this 
system  was  estimated  to  be  48  hours  which,  again  as  in  Example  1,  was  based  on  historical  data 
for  similar  type  systems  and  on  lower  level  testing  for  the  system  under  development.  For  this 
developmental  program,  it  was  assumed  that  a  growth  rate  (a)  of  0.30  would  be  appropriate  for 
reliability  growth  purposes.  Now,  for  this  example,  suppose  we  want  to  determine  the  total 
amount  of  system  level  test  time  (T)  such  that  the  Type  I  or  producer  risk  for  the  program 
idealized  reliability  growth  curve  is  0.20;  i.e.,  the  probability  of  not  demonstrating  the  TR  of  100 
hours  with  80  percent  confidence  is  0.20  for  the  final  MTBF  value  (M(T))  obtained  from  the 
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Now,  .0  detennino  U.e  test  time  T  which^« 
we  first  select  an  initial  value  of  T  ai^  as  in  ^  M(T)/TR  and  E(N)  as  entnes  m 

failures  (E(N))  »"  P')„  Zdouble  lineaa  interpolation,  the  probability 

following  iterative  results: 


T 

M(T) 

E(N)  _ 

P(A) 

■^000 

117.4 

36.5 

<0.412 

4000 

128.0 

44.6 

<0.610 

5000 

136.8 

52.2 

<0.793 

5500 

140.8  n 

55.8  n 

0.815 

5400 

140.0 

55.1 

0.804 

5300 

139.2 

54.4 

0.790 

5350 

139.6 

54.7 

0.796 

5375 

139.8 

54.9 

0.800 

j  .  hnur<i  to  be  the  required  amount  of  system 

level  tesu[m?such?hauhe  T^e"!  or  pmiucer  risk  for  the  program  idealized  growth  curve  is 

0.20. 

1 1  S  Summary  The  concepts  of  an  operating  characteristic  (OC)  analysis  have  been 

extended't;  the  reliabili^  growti.  " 

Statistical  risks  have  been  expressed  m  term  confidence  level,  these  risks  have 

duration,  and  reliability  requirement.  p  ^  failures  during  the  growth  test  and  the 

The  methodology  developed  and  illustrated  in  this  section  should  be  “f 
a„alysts“bfe  fo?^— g  maiistic 
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2.2  Subsystem  Level  Planning. 

2.2.1  Subsystem  Reliability  Growth.  This  material  is  based  on  Reference  [4]. 

2.2.1.1  Benefits  and  Special  Considerations.  Conducting  a  subsystem  reliability 
growth  program  prior  to  the  start  of  system  level  testing  can  - 


•  reduce  the  amount  of  system  level  testing, 

•  reduce  or  eliminate  many  failure  mechanisms  (problem  failure  modes)  early  m  the 
development  cycle  where  they  may  be  easier  to  locate  and  correct, 

•  allow  for  the  use  of  subsystem  test  data  to  monitor  reliability  improvement. 


•  increase  product  quality  by  placing  more  emphasis  on  lower  level  testing  and 

•  provide  management  with  a  strategy  for  conducting  an  overall  reliability  growth 
program. 


Thus,  subsystem  reliability  growth  offers  the  potential  for  signiftcant  savings  in  testing  cost. 


presence 
adhere  as 


•  Potential  high-risk  interfaces  need  to  be  identified  and  addressed  through  joint 
subsystem  testing, 

.  Subsystem  usage/test  conditions  need  to  be  in  conformance  with  the  proposed  system 
level  operational  environment  as  envisioned  in  the  Operational  Mode 
Summary/Mission  Profile  (OMS/MP), 

.  Failure  Defmitions/Scoring  Criteria  (FD/SC)  formulated  for  each  subsystem  need  to 
be  consistent  with  the  FD/SC  used  for  system  level  test  evaluation. 


2  212  Overview  of  Subsystem  Reliability  Growth  Planning  Model  -  SSPLAN.  The 
subsystem'rdiability  growth  planning  model,  SSPLAN,  provides  the  user  with  a  means  to 

"mbsystemttog  plL  for  demonstrating  a  system  r^ean  time  b 

goal  orior  to  system  level  testing.  (The  MTBF  goal  is  also  referred  to  as  the  MTBF  objective 
fMTBFobj) )  In  particular,  the  model  is  used  to  develop  subsystem  reliability  ^owth  plying 
curves  that,  with  a  specified  probability,  achieve  a  system  MTBF  objective  with  a  specified 
confidence  level.  More  precisely,  associated  with  the  subsystem  M^BFs  ^wing  a  ong  a 
planned  growth  curves  for  given  subsystem  test  durations  is  a  P^bability;  this  is  temed  the 
probability  of  acceptance  (Pa),  the  probability  that  the  system  .j.. 

demonstrated  at  the  specified  confidence  level.  The  complement  of  Pa,  -  a,  ,  •  ■  ^ 

producer’s  (or  contractor’s)  risk:  the  risk  of  not  demonstrating  the  system  MTBF  objective 
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specified  confidence  level  when  the  subsystems  are  growing  along  their  target  growth  curves  for 
the  prescribed  test  durations.  Note  that  Pa  also  depends  on  the  fixed  MTBF  of  any  non-growth 
subsystem  and  on  the  lengths  of  the  demonstration  tests  on  which  the  non-growth  subsystem 
MTBF  estimates  are  based. 


SSPLAN  estimates  Pa  for  a  given  value  of  the  final  combined  growth  subsystem  MTBF 
(MTBFg  svs)  by  simulating  the  reliability  growth  of  each  subsystem  and  calculating  a  statistica 
lower  confidence  bound  (LCB)  for  the  final  system  MTBF  based  on  the  growth  and  non-growth 
subsystem  simulated  failure  data.  If  the  system  LCB,  at  the  specified  confidence  level,  meets  or 
exceeds  the  specified  MTBF  goal,  then  the  trial  is  labeled  a  success.  SSPLAN  runs  as  many  as 
5000  trials  and  estimates  Pa  as  the  number  of  successes  divided  by  the  number  of  tnals. 


One  of  the  model’s  primary  outputs  is  the  growth  subsystem  test  times.  If  the  growth 
subsystems  were  to  grow  along  the  planning  curves  for  these  test  times  then  the  prob^ihty 
would  be  Pa  that  the  subsystem  test  data  demonstrate  the  system  MTBF  objective,  MTBFobj,  a 
the  specified  confidence  level.  The  model  determines  the  subsystem  test  times  by  using  a 
specified  fixed  allocation  of  the  combined  final  failure  intensity  to  each  of  the  individual  growth 

subsystems. 

As  a  reliability  management  tool,  the  model  can  serve  as  a  means  for  prime  contractors  to 
coordinate/integrate  the  reliability  growth  activities  of  their  subcontractors  as  part  of  their  overall 
strategy  in  implementing  a  subsystem  reliability  test  program  for  their  developmental  systems. 

2.2.1. 3  List  of  Notation.  There  are  some  variant  terms  in  the  following  parameter  list  to 
show  that  the  form  of  some  parameters  depends  on  the  context  in  which  they  are  used.  For 
example,  T,  ,  and  ,  indicate,  respectively,  that  time  may  be  used  generically,  specifically 

for  non-growth  subsystem  i  and  specifically  for  growth  subsystem  i.  Also,  for  notational 
convenience,  several  parameters  that  can  vary  by  subsystem  are  sometimes  wntten  without  a 
subsystem  subscript.  However,  subscripts  are  used  where  required  for  clanty. 


t 

T 

F(t) 

E[F(t)] 

X 

P 

a 

^1 

MTBF 

M, 

ms 

/KO 


subsystem  test  time 

total  subsystem  test  time  (0<t  <T^ 

total  number  of  subsystem  failures  by  time  t 

expected  number  of  subsystem  failures  by  time  t 

AMSAA  model  scale  parameter  [A  >  O)  for  growth  subsystem 

AMSAA  model  shape  (or  growth)  parameter  {P>  0)  for  growth 

subsystem 

growth  rate  {^a  =  \-P),  (0<<2r<l) 

initial  time  period  for  subsystem  growth  test,  ( r,  >  0 ) 

Mean  Time  Between  Failure 
initial  average  MTBF  over  interval  ( 0,/^  ] 
initial  average  failure  intensity  over  interval  ( 0,r,  ] 
management  strategy,  (0  <ms  <1) 
instantaneous  failure  intensity  at  time  t,  [/?(^/>0j 
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Q  O  . 


M(t) 
MTBFobj  - 
Pa 

LCB 


Tdj 

To, 

Tmax.i 

«D.i 


n 


G  .• 


Mo, 

PD.i 

Mo, 

Po.  I  (T G,  i) 


P  D, 

po.i  (To.i) 
Xdf 

Psrs 


a 

M 


SYS 


M, 


G.SYS 


M 

”  D.SYS 

Td.SYS 

mle 


Mo, 

Mo, 


instantaneous  MTBF  at  time  t 

system  MTBF  objective  to  be  demonstrated  with  confidence  y 
probability  of  acceptance  associated  with  demonstrating  MTBFobj 

lower  confidence  bound 

demonstration  (non-growth)  test  data  or  estimator  subscript 
growth  test  data  or  estimator  subscript 
subsystem  index  number 

total  amount  of  demonstration  or  “equivalent  demonstration 

(non-growth)  test  time  for  subsystem  i 

total  amount  of  growth  test  time  for  subsystem  i 

specified  maximum  allowable  growth  test  time  for  subsystem  i. 

Thus  Tg.  i<TMAX,i 

number  of  failures  during  a  demonstration  test  of  length  Td.i  for  a 
non-growth  subsystem  i.  Also,  number  of  “equivalent  demonstration 
failures  for  growth  subsystem  i  during  growth  test 
number  of  failures  during  a  test  time  To.i  for  a  growth 

subsystem  i 

demonstration  (constant)  MTBF  for  non-growth  subsystem  i 
equals  A/^!, 

Final  MTBF  for  growth  subsystem  i 
equals 

denotes  an  estimate  when  placed  over  a  parameter 
estimate  of  Poj 
estimate  of  po.i  (Tg.O 

chi-squared  random  variable  with  “df  ’  degrees  of  freedom 
final  system  failure  intensity 

total  failure  intensity  contribution  of  growth  subsystems  to  p^^^ 
fraction  of  Po,sYs  allocated  to  growth  subsystem  i 
final  system  MTBF 

final  MTBF  of  combined  growth  subsystems,  i.e.,  MG.sys  =  /?a'^ 
system  demonstration  “equivalent”  number  of  failures 
system  demonstration  “equivalent”  test  time 
maximum  likelihood  estimate 

symbol  for  “distributed  as”  a  specified  random  variable 
subsystem  i  MTBF  estimate  of  demonstration  or 
-  “equivalent  demonstration”  MTBF 
subsystem  i  mle  for  final  MTBF  of  growth  subsystem 
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M  SYS  estimate  of  final  system  MTBF 

/  specified  confidence  level  for  demonstrating  MTBFobj 

chi-squared  1  OOy  percentile  point  for  df  degrees  of  freedom 


Pi 

PsYS 

K 

LCB 

LCB,,^r 

(  )/ 
c 

^Total 

Ci[TD,i] 

Ci[PG.i(TG.i)] 

G,sys  ')i^EW 
G.sys 
G,sys 

(^.), 

(Pdc 


fUB 


f  LB 


WB 


/GOAL 


estimate  of  final  subsystem  i  failure  intensity 
estimate  of  final  system  failure  intensity 
number  of  subsystems 

subsystem  i  LCB  at  /  confidence  level  from  demonstration  data 
subsystem  i  LCB  at  /  confidence  level  fi'om  growth  data 
cost  per  failure  for  subsystem  i 
cost  per  hour  for  subsystem  i 
total  testing  cost 

cost  contribution  of  non-growth  subsystem  i  to  Cfotai  as  a  fimction  of  To.i 
cost  contribution  of  growth  subsystem  i  to  Crotai  as  a  function  of  PG,i(TG,i) 
new  value  of  Me,  syi  to  use  in  search  routine 

lower  bound  for  Me,  sys 

upper  bound  for  Me 

estimated  associated  with  {Mc,sys)i^g 

estimated  associated  with  (M<j 

desired  P^ 


2.2.2  SSPLAN  Methodology. 


2.2.2.1  Model  Assumptions.  The  SSPLAN  methodology  assumes  that  a  system 

may  be  represented  as  a  series  of  >  1  independent  subsystems.  (The  theory  allows  for  K  =  \ 
but  the  current  computer  implementation  requires  K>2.) 


System 

= 

Subsystem  1 

"f"  ... 

Subsystem  K 

This  means  that  a  failure  of  any  single  subsystem  results  in  a  system  level  failure  and  that  a 
failure  of  a  subsystem  does  not  influence  (either  induce  or  prevent)  the  failure  of  any  other 
subsystem.  SSPLAN  allows  for  a  mixture  of  test  data  from  growth  and  non-growth  subsystems, 
but  in  its  current  implementation,  at  least  one  growth  subsystem  is  required  to  run  the  model. 
The  model  utilizes  the  following  assumption  for  the  growth  subsystems; 

•  The  number  of  failures  occurring  over  a  period  of  test  time  follows  a 
nonhomogeneous  Poisson  process  (NHPP)  with  mean  value  function 

£’[f(0]  =  {A,jB,t>0)  (1) 
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where  E[F(t)]  is  the  expected  number  of  failures  by  time  t,  Z  is  the  scale  parameter  and  /?  is  the 

growth  (or  shape)  parameter.  The  parameters  X  and  P  may  vary  from  subsystem  to  subsystem 
and  will  be  subscripted  by  a  subsystem  index  number  when  required  for  clarity.  Non-growth 
subsystems  are  assumed  to  have  constant  failure  rates. 

2.2.2.2  Mathematical  Basis  for  Growth  Subsystems. 

2.2.2.2.1  Initial  Conditions.  The  power  function  shown  in  (1)  together  with  the  initial 
conditions  described  in  this  section  provide  a  framework  for  a  discussion  of  the  way  SSPLAN 
develops  reliability  growth  curves.  Together  they  provide  a  starting  point  for  describing  each 
growth  subsystem’s  MTBF  as  a  function  of  the  parameters  /I ,  /?  and  t.  Since  X  is  not 
convenient  to  directly  work  with  for  planning  purposes,  we  shall  relate  to  an  initial  or  average 
subsystem  MTBF  over  an  initial  period  of  test  time.  First,  we  note  that  the  growth  parameter, 

P,  is  related  to  the  grovi^h  rate,  a ,  by  the  following: 

P  =  \- a  (;^>0)  (2) 

For  planned  growth  situations,  or  must  be  in  the  interval  (0,1).  Additional  guidance  on  choosing 
a  may  be  gained  from  Ellner  &  Trapnell  [5]. 

The  initial  conditions  for  the  model  consist  of: 

•  an  initial  time  period,  t,  (for  example,  the  amount  of  planned  test  time  prior  to  the 
implementation  of  any  corrective  actions),  and 

•  the  initial  MTBF,  M, ,  representing  the  average  MTBF  over  the  interval  ( 0,t/  ] . 
From  this,  note  that 


^  (M,>0)  (3) 

M, 

is  the  average  failure  intensity  over  the  interval  ( 0,t,  ] .  The  fact  that  (1)  must  be  consistent  with 
the  initial  conditions  allows  the  scale  parameter,  /i ,  to  be  expressed  in  terms  of  planning 
parameters  U,  Mi,  and  a.  To  do  so,  note  the  expected  number  of  failures  by  time  t,  is: 

£[F(r,  )]  =  Ajt,  (t/>0)  (4) 

Using  (1),  we  see  that  the  expected  number  of  failures  by  time  is  also  given  by 

F[F(r,  )]  =  it/  (5) 
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By  equating  (4)  and  (5)  and  by  using  the  relationship  tr  =  1  -/?  front  (2).  an  expression  for  i 
may  be  developed: 


-  •  ~  M, 


(6) 


In  addition  to  using  both  M,  and  t,  as  initial  growth  subsystem  input 

~HiSF=S!S=“' 

addressed  in  the  following  discussion. 

raSSrTaHSSStr 

our  assumptions,  the  number  of  failures  that  occur  over  the  mittal  time  penod  r,  is  Po 

distributed  with  expected  value  Thus 


0.95 


\-e 


-(  ) 


msy-tf 


(0  <ms  <1)  (7) 


growth  subsystems. 

2  2  2  2  2  Failure  Intensity  and  Mean  Time  Between  Failures  -  MTBF.  The 
derivative  wifh  respect  to  time  of  the  expected  number  of  failures  function  (1)  is; 


p{i)  =  w 

The  function  fki)  represents  the  instantaneous  failure  intensity  at  time  t.  The  reciprocal  of  M<) 
is  the  instantaneous  MTBF  at  time  t; 


M(t)  =  ^ 


(9) 


subsystem  reliability  growth. 
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Figure  5.  Reliability  Growth  Based  On 
AMSAA  Continuous  Tracking  Model. 


,.2.2.3  Ma—ca.Bas.f.rNo^^^^ 

failure  rate  assumption,  *e  input  *  demonstration  test,  T,  upon  which  the  constant 

fixed  reliability  estimate,  M,  and  the  length  ot  me  a 

MTBF  estimate  is  based. 


of 


S4  i  t>r  lo 

2.2.2.4  Algorithm  ^"0 

purely  analytic^  meftods  SSPL^  ^  ^  ^hU  estimate  of  P*  is 

Si;Cin^  — .ion  a  large  number  of  tnals. 

Using  the  parameters  that  have 

™odel  generates  -test  dam"  for  ea*  -bsy^J- ^r  each  snnu^^ 

data  required  to  produce  an  estim  rorresoonding  to  the  set  of  subsystems  that  compnse 

The  model  then  uses  a  method  developedJor  discmte^d^a(t^^^^ 

Method)  to  “roll  up”  the  subsystem  data  to  j  confidence  bound  (LCB)  for 

reliability  at  a  specified  method  to  be  able  to  handle  a  mix 

the  final  system  MTBF.  Lbsystems,  the  model  first  converts  aU  growth 

of  test  data  from  both  growth  md  "  §  demonstration  (D)  test  time  and 

(G)  subsystem  test  data  to  an  equiv  This  conversion  process  is  done  so  that  all 

'•ecuivalLf  number  of  demonstraUon  failures,  Th^  “on  p  ,„„figu„tion 

subsystem  results  are  expressed  in  ^  ^™".kne  Ind  the  equivalent 

(non-growth)  test  data.  (The  equivalen  and  the  number  of  failures 
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demonstration  testing  may  be  used  to  compute  the  system  reliability  LCB  for  the  combination  of 
“converted”  growth  3nd  non-growth  test  data. 

SSPLAN  can  run  as  many  as  5000  trials.  For  each  simulation  trial,  if  the  LCB  for  the 
final  system  MTBF  meets  or  exceeds  the  specified  system  MTBF  objective,  then  the  tnal  is 
termed  a  success.  An  estimate  for  the  probability  of  acceptance  is  the  ratio  of  the  number  of 
successes  to  the  number  of  trials. 

The  algorithm  for  estimating  the  probability  of  acceptance  is  described  in  greater  detail 
by  expanding  upon  the  following  four  topics: 

•  generating  “test  data”  estimates  for  growth  subsystems 

•  generating  “test  data”  estimates  for  non-growth  subsystems 

•  converting  growth  subsystem  data  to  “equivalent”  demonstration  data 

•  using  the  Lindstrom-Madden  method  for  computing  system  level  statistics 

2.2.2.4.1  Generating  Estimates  for  Growth  Subsystems.  There  are  two  quantities  of 
interest  for  each  growth  subsystem  for  each  trial  of  the  simulation  - 

•  the  total  amount  of  test  time,  7J; , ,  and 

•  the  estimated  failure  intensity  at  that  time,  Pq  i  ,• )  • 

To  calculate  , ,  note  that  from  the  initial  input  conditions  we  have  values  for  the 
growth  parameter,  (using  (2)),  and  the  scale  parameter,  X-  (using  (3)  and  (6)).  Also,  note  that 
the  final  growth  subsystem  MTBF,  ,  can  be  calculated  by  dividing  the  final  MTBF  of  the 
combined  growth  subsystems,  the  subsystem  failure  intensity  allocation  uj. 

Equations  (8)  and  (9)  can  then  be  combined  and  rearranged  to  solve  for  : 


T  = 


(10) 


To  generate  the  estimated  failure  intensity,  Pc,i^G,i)’  th®  model  uses  X ,  7^ ,  and  (1) 

with  t  =  To.i  to  calculate  a  Poisson  distributed  random  number,  >  which  serves  as  an  outcome 
for  the  number  of  growth  failures  during  a  simulation  trial.  The  model  then  generates  a  chi- 
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squared  random  nunrber  with  2n,,,  degrees  of  freedom  and  uses  relation  (1 1)  below  referenced 
in  Crow  [6]  for  obtaining  a  random  value  from  the  distribution  for  the  estimated  growth 
parameter,  conditioned  on  the  number  of  growth  failures,  n^.,,  during  the  tnah 

1  (11) 

d'l.., 

Note  0  is  obtained  from  the  initial  input  and  (2).  One  can  show  .  and  the  maximum 
likelihood  estimates  (mle’s)  for  A,  and  p  satisfy  the  following: 

n,..  =  XtI  (i,M.,>0)  (12) 


In  light  of  equation  (1),  this  result  is  not  surprising. 
Using  mle's  for  the  parameters  in  (8)  yields. 

PcATc,)  = 


(13) 


Rearranging  terms  in  (13)  we  obtain; 

PcATo,  ) 


i  Tj,  P 

To, 


(14) 


Substituting  (12)  into  (14)  we  conclude: 


(15) 


Thus  using  no  i  and  the  corresponding  conditional  estimate  for  p  generated  from  (11),  an 
estimate  for  the  failure  intensity,  can  be  obtained  for  each  growth  subsystem  for  each 

trial  of  the  simulation.  Note  the  same  value  for  To  i  is  used  on  all  the  simulation  tnals. 

2.2.2.4.2  Generating  Estimates  for  Non-growth  Subsystems.  There  are  two  quantities 
of  interest  for  each  non-growth  subsystem  for  each  tnal  of  the  simulation  - 


•  the  total  amount  of  test  time,  , ,  and 

•  the  estimated  failure  intensity,  Po.^d.^- 


The  total  amount  of  test  time,  ,  is  an  input  planning  parameter  that  represents  the 

length  of  the  demonstration  test  on  which  the  non-growth  subsystem  MTBF  estimate  U  based 
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•1  •  ,  ^  ('v  \  tVie  model  first  c^lciil3-tes  (this  is  done 

To  generate  the  estimated  failure  intensity, 

only  once  for  each  non-growth  subsystem  in  SSPLAN)  the  expecten  nnm 


4F(rn,)l  = 


(16) 


D,i 


where  M„.  is  an  input  planning  parameter  repr^enting  the  con^ant  MTBF  for  the 

subsystem.'  The  expected  nmnber 

S— :  for  the  number  of  failures  during  a  simulation  trial. 

An  estimate  for  the  failure  intensity  follows: 


(r  )  = 

Pd,\  V  ^  DJ  )  J' 

^  D,i 


(17) 


1 .  converting  all  growth  subsystem  data  to  ••equivalent"  demonstration  data,  that  is,  dala 

from  a  fixed  configuration.  These  data  consist  ot. 

.  r„,  -  subsystem  i  equivalent  demonstration  test  time  and 

.  subsystem!  equivalent  demonstration  number  of  failures 

2.  using  the  Lindstrdm-Madden  method  to  obtain  system  level  statistics  for  calculating 
the  LCB  for  the  system  MTBF. 

S:  na^Xthe  demLttation  data  and  the  growth  data  must  yield. 


1.  the  same  subsystem  MTBF  point  estimate. 


Mo,  = 


(18) 


2.  and  the  same  subsystem  MTBF  lower  bound  at  a  specified  confidence  level  y: 

,/-D  (19) 

LCB„,,  =  iCBo,i,r 

Starting  with  the  left  side  of  the  second  equivalency  relmionship.  (19),  note  that  the  lower 
confidence  boLd  formula  for  time-truncated  demonstration  testing  i  . 
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(20) 


LCB  —  2 


27, 


D.i 


X  2no,i+2,/ 


where  is  the  demonstration  test  time,  n^  i  is  the  demonstration  number  of  failures,  /  is  the 

specified  confidence  level  and  zlno<^2.r  a  chi-squared  100  y  percentile  point  with  2/1^ ,  +  2 

degrees  of  freedom.  Using  an  approximation  equation  developed  by  Crow,  the  lower  confidence 
bound  formula  for  growth  testing  (the  right  side  of  (19))  is: 


LCB 


G,i.r 


^G.i  ^  Gj 

X  /ic,,+2,r 


(21) 


where  is  the  number  of  growth  failures  during  the  growth  test,  is  the  mle  for  the 
MTBF  and  is  a  chi-squared  lOOy  percentile  point  with  +  2  degrees  of  freedom 

Since  we  want  (20)  and  (21)  to  yield  the  same  estimate,  we  begin  by  equating  their 
denominators: 


2^0,/  +  2  —  fiQ  ■  +2  => 

Equating  numerators  fi'om  (22)  and  (23)  yields: 


«£),/  =• 


(22) 


27’d,/ 


T  = 

^  D,i 


(23) 


Thus  (19)  holds  for  no.i  and  To.i  given  by  (22)  and  (23)  respectively  in  terms  of  the  simulated 
growth  test  data. 

Dividing  (23)  by  (22): 


T 

^  D.i 


=  ^GJ 


"D.i 


(24) 


Thus  (18)  is  also  satisfied. 

By  (23)  we  obtain: 

ftri 

fjn  _  Cf,/ 

'2.  pc.  I  (7c.  i) 


(25) 
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From  (13)  we  have: 


(26) 


Multiplying  both  numerator  and  denominator  of  (26)  by  7^ , ,  replacing  the  estimate  of  the 
expected  number  of  failures  (in  the  denominator)  by  the  observed  number  of  growth  failures  and 
canceling  the  term  ■  in  the  numerator  and  denominator  yields. 


(27) 


SSPLAN  uses  (22)  and  (27)  in  converting  growth  subsystem  data  to  equivalent  demonstration 
data. 

2  2  2  4  3  2  Using  the  Lindstrom-Madden  Method  for  Computing  System  Level 
Statistics.*  A  continuous  version  of  the  Lindstrom-Madden  method  for  discrete  subsystems  is 
used  to  compute  an  approximate  lower  confidence  bound  (LCD)  for  the  final  system  MTBF  from 
subsystem  demonstration  (non-growth)  and  “equivalent”  demonstration  (converted  growth)  data. 
The  Lindstrdm-Madden  method  typically  generates  a  conservative  LCB,  which  is  to  say  t  e 
actual  confidence  level  of  the  LCB  is  at  least  the  specified  level.  It  computes  the  following  four 

estimates  in  order: 

1.  the  equivalent  amount  of  system  level  demonstration  test  time.  (Since  this  estimate  is 
the  minimum  demonstration  test  time  of  all  the  subsystems,  it  is  constrained  by  the 
least  tested  subsystem.) 

2.  the  estimate  of  the  final  system  failure  intensity,  which  is  the  sum  of  the  estimated 
final  growth  subsystem  failure  intensities  and  non-growth  subsystem  failure  rates 

3.  the  equivalent  number  of  system  level  demonstration  failures,  which  is  the  product  of 
the  previous  two  estimates. 

4.  The  approximate  LCB  for  the  final  system  MTBF  at  a  given  confidence  level,  which 
is  a  fimction  of  the  equivalent  amount  of  system  level  demonstration  test  time  and  the 
equivalent  number  of  system  level  demonstration  failures. 

In  equation  form,  these  system  level  estimates  are,  respectively: 


=  minTo 


for  i  =  1...K 


(29) 
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where  Pi  - 


M 


and  Mo ,  =  the  demonstration  or  equivalent  demonstration  MTBF  estimate 


0,1 


for  subsystem  i. 


N 


D.Sys 


=  '^D,SYS  ^  PSYS 


LCB^  = 


IT, 


D.SYS 


(30) 


(31) 


112  5  Calculation  of  Testing  Costs.  SSPLAN  can  be  used  to  calculate  the  cost  of 
carrying;;t;subsystem  reliability 

foe^aSl r^nction  of  the  number  of  failures  and  all  costs  that  are  a  function 
of  time,  as  shown  respectively  in  the  following  formula; 


C 


Total 


^ {^[F^rolxlC^  ),  +  T.x{Cr  ),  } 

ie{all  subsystems} 


(32) 


In  (32),  for  each  subsystem  i,  T,  denotes  the  amount  of  test  time,  e{f{T,  )]  is  the  expected 

nnmberoffailuresbytime  T„(C,\ is  the  cost  per  failure,  and  (  C, ),  P'' ° 

time  (usually  per  hour).  So,  the  total  testing  cost,  C,„, ,  is  the  sum,  over  a  su  sys  , 
costs  associated  with  testing  each  subsystem. 

Once  again,  it  is  useful  to  treat  growth  and  non-growth  subsystems  separately. 

J  2  2  5.1  Calculating  Cost  for  Growth  Subsystems.  For  a  given  ^lution,  we  cm 
calculate  the  cost  contribution  to  Cr.»i  of  a  growth  subsystem  i  in  temy>  o.,  m  ^grow 

parameters  XH.P,  by  directly  using  (32)  with  T,  =  To.,.  Noteby(l),  -  Vc., 

Alternately  we  can  express  this  cost  in  tetms  of  the  achieved  subsystem  failure 

Pg!,TtS  and  X„  P, .  To  write  the  cost  equation  in  terms  of  the  subsystem  failure  tntens.ty,  we 

begin  by  obtaining  an  expression  for  ,  from  (8); 


pM  =  (i,.A.ro„>o)  (33) 


Isolating  the  To.i  term  on  one  side  of  (33)  yields: 

PgJ  (  ^G.i  ) 


(34) 


i,  A- 


Raising  both  sides  of  (34)  to  the  l/(  A  f )  power. 
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(35) 


(Note  /?,.  1  since  subsystem  i  is  a  growth  subsystem.) 

Substituting  from  (2)  yields  the  following  intermediate  result: 

=  [/’wlJ'a,,  )]  (0<«,<1)  (36) 

Now,  to  obtain  an  expression  for  £'[f^(rc,i)],  we  begin  with  (1): 

(37) 


Substituting  for  7^  ,•  from  (36)  yields: 


A 


£  [  F,(  Tga  )  ]  =  [  Pg,  (  Tgj  )  ]  ^  ^  A  1 

Rearranging  terms  in  (38)  yields: 

(-5 


f) 


(38) 


E[Fi{  Tg. ( )  ]  =  ^  ’  [ pG,i  ( T’c,-  )  ]  ^  ^  AI 


Pi 


(39) 


Finally,  the  cost  contribution  in  (32)  of  growth  subsystem  i  can  be  expressed  in  terms  of  its 
failure  intensity  using  (39)  and  (36): 


c,[pM  = 


(40) 


2.2.2.5.2  Calculating  Cost  for  Non-growth  Subsystems.  To  obtain  the  cost 
contribution  of  a  non-growth  subsystem,  we  use  (16)  to  express  E[Fi{TD.i)]  in  terms  of 


and 
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^  'T  ^ 
^  D,i 

V  ^D.i  J 


{Cp  ),■  +  Td.  i  ( Cj.  ),• 


(41) 


C,  1  = 

where  =  /?D,i 

2  2.2.6  Methodology  for  a  Fixed  Allocation  of  Subsystem  Failure  Intensities.  The 

methodology  utilizes  a  fixed  allocation,  as,  of  po.srs  to  each  growth  subsystem  i.  Thus 

Dr  (To  i)  =  For  this  allocation,  SSPLAN  first  determines  if  a  solution  exi^s  that 

s^atisfies  the  criteria  given  by  the  user  during  the  input  phase.  Specifically,  SSPLAN  checks  to 

see  if  the  desired  probability  of  acceptance  can  be  achieved  with  the 

allocations  and  maximum  subsystem  test  times.  If  a  solution  does  exist,  SSPL  p 

to  find  the  solution  that  meets  the  desired  probability  of  acceptance  within  a  small  positive 

number  epsilon. 

2.2.2.6.1  Determining  the  Existence  ota  Solution.  To  determine  if  a  solution  is 
possible  SSPLAN  uses  (8)  and  (9)  for  each  subsystem,  with  T  set  to  the  subsystem  s  maximum 
S  drne  to  calculate  the  maximum  possible  MTBF  for  each  subsystem.  The  maximum 
subsystem  MTBF  is  multiplied  by  its  failure  intensity  allocation  to  i, 

system  MTBF.  For  example,  if  a  subsystem  can  grow  to  a  maximum  MTBF  of  1000  houm  and 
has  a  failure  intensity  allocation  of  0.5  (that  is,  its  final  failure  intensity  accounts  for  half  of  the 
toTal  final  failure  intLsity  due  to  all  of  the  growth  subsystems),  then  that  pamcular  subsystem 
Slir' the  combined  ^owth  subsystem  maximum  MTBF  to  500  hour.  Tn  other  words,  the 
maximum  MTBF  to  which  the  growth  portion  of  the  system  can  grow,  MTBFc  ^y, ,  is 
minimum  of  the  products  (subsystem  final  MTBF  multiplied  by  the  subsystem  failure  intensity 
allocation)  from  among  all  the  growth  subsystems. 

The  probability  of  acceptance,  P, ,  is  then  estimated  using  If  the  estimated 

is  less  than  the  desired  ,  then  no  solution  is  possible  within  the  limits  of  estimation 
precision  for  P, ,  and  SSPLAN  will  stop  with  a  message  to  that  effect. 

2.2.2.6.2  Finding  the  Solution.  On  the  other  hand,  if  the  estimated  P^  is  greater  than  or 
equal  to  the  desired  P, ,  then  a  solution  exists.  If,  by  chance,  the  desired  P,  has  been  met 
(within  a  small  number  epsilon)  then  SSPLAN  will  use  MTBF^  ,^,  as  its  solution.  It  is  more 
likely,  however,  that  the  estimated  P^  corresponding  to  AiTBP^  ^y,  exceeds  the  requirement, 
meaning  that  the  program  resulting  in  MTBFo.  sys  contains  more  testing  than  is  necessary  to 
achieve  the  desired  P, .  SSPLAN  proceeds,  then,  to  find  a  value  for  MTBFo.  sys  that  meets  the 

desired  within  epsilon. 


To  save  time,  P^  is  initially  estimated  using  a  reduced  number  of  iterations  equal  to  one 
tenth  of  the  requested  number.  As  soon  as  the  estimated  P^  approaches  the  desired  P, ,  the  full 
number  of  iterations  is  used. 
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For  a  given- fixed  failure  intensity  allocation,  P,  increases  as  MTBF  increases.  Every 

^  ^  rifrpliahilitv  growth  curves,  and  thus  a  unique  . 

value  of  MTBFg,  sys  determines  a  uniqu  •  ^  tn  the  desired  P  SSPLAN  first  finds  the 

TO  find  the  set  of  growfi,  curve  test  times  that  give  nse  to  the  destmd  P  SSP 

upper  and  lower  bounds  for  AfraFo,„  •  The  initial  upper  bound  tor  MTBF, 

fid  in  verifying  the  existence  of  a  solution;  this  j  Lllol  bound  for 

uthf  tbased  on  the  maximum  test  times  inputted  by  the  user),  mem  ,  ^  , 

UTBf""  s  chosen  arbitrarily;  if  the  value  chosen  results  in  a  P,  that  is  higher  than  the  desired 
f  I:;  Tower  bound  for  M^F,^  is  successively  decrease  until  the  resulting  P.  is  less 

Ui;;the desired  P.  A. poinl, uppe-d lower boimds for 

u, •  u  ^  Q^PT  AN  uses  a  linear  interpolation  to  find  the  value  oi  -y  t. 

established,  and  SSPLAN  uses  a  n  f  MTBF,,,,  is 

consistent  with  Reference  [4]). 


{MTBFg,,,  1^^-{MTBF„„} 
(  MTBFg,  sys  — (  MTBFq 


>sys  )lb  _ 


(  )goal  ( 

{Pa  )ub  -{Pa  )lb 


(42) 


where  (UTBF,  )  and  (MTBFo.  refer  to  the  upper  and  lower  bounds,  respective^  for 

■  (  7”)  "and  (  F.  )  refer  to  the  estimated  P.  values  associated  with  each  of  the 

MTBFg,,,  ,[P,)gs^^y  1  of  MTBFg,,, 

preceding  MTBFg,,,  values,  respectively;  and  [MTBFg,,, 

to  be  used  in  the  search  algorithm. 

V  A  )goal  ’  u  j  •  /I  p  then  ( MTBF  )  becomes  the  new  upper 

— v.?:r::r:.r  '■ 

the  lower  and  upper  bounds  on  MTBFo..„  are  within  epsilon  of  each  ot  . 
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3  reliability  growth  tracking 


3.,  ,„.r«<.uc«oo.  This  s..ion  contains 

3,.l  Deflnitio.  and  Objectives  otR« 

tracking  ts  a  process  that  allows  numerical  measure  of  the  system 

rdSabulrdSn/rdeveL^i 

growth  tracking  include; 

.  determining  if  system  reliability  is  increasing  with  time  (i.e.,  growth  is  occumng)  and 
to  what  degree  (i.e.,  growth  rate),  and 

.  estimating  the  mst'pSs^  This  latter  estimate  is 

configuration. 

Reliabihty  growth  trackmg  ^lo^^^^^^^^ 

r„rpri;rS2ky  growth,  the^^^ 

representative  of  the  current  configuratio  y  g  jnay  be  limited  so  that  an 

data,  which  would  best  represent  the  ^  besufficient  for  a  valid  determination  of 

estimate  based  upon  the  recent  data  ’  ^Q^th’tracking  may  offer  a  viable  method  for 

reliability.  Because  of  this  situation,  .  •  <iemonstrated  reliability  estimate  for 

represents  the  combined  test  data. 

3.1.2  Managerial  Role.  The  role  of  management  in  the  reliability  growth  tracking 
process  is  twofold: 

•  ru,r;m"mcc" 

demonstrated  reliability  values. 

TO  achieve  reliability  goals,  it  rnXmoSToZISc"  ^ 

problems  during  the  conduct  o  t  e  therefore,  that  periodic  assessments 

changes  can  be  funded  and  implemen  _  program’  (usually  at  the  end  of  a  test  phase)  and 

rp3  “XS'X.'-i  ~ 
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values  will  suggest  whether  the  development  program  is  progressing  as  planned,  better  than 
planned,  or  not  as  well  as  planned.  Thus,  tracking  the  improvement  in  system  reliability  through 
quantitative  assessments  of  progress  is  an  important  management  function. 

3.1.3  Types  of  Reliability  Growth  Tracking  Models.  Reliability  growth  tracking 
models  are  distinguished  according  to  the  level  at  which  testing  is  conducted  and  failure  data  are 
collected.  They  fall  into  two  categories:  system  level  and  subsystem  level.  For  system  level 
reliability  growth  tracking  models,  testing  is  conducted  in  a  full-up  integrated  nianner,  failure 
data  are  collected  on  an  overall  system  basis,  and  an  assessment  is  made  regarding  the  system 
reliability.  For  subsystem  level  reliability  growth  tracking  models,  the  subsystems  are  tested  and 
the  failure  data  are  collected  on  an  individual  subsystem  basis  —  the  subsystem  data  are  then 
“rolled  up”  to  arrive  at  an  estimate  for  the  demonstrated  system  reliability. 

System  level  reliability  growth  tracking  models  are  further  classified  according  to  the 
usage  of  the  system.  They  fall  into  two  groups  ~  continuous  and  discrete  models  -  and  are 
defined  by  the  type  of  outcome  that  the  usage  provides.  Continuous  models  are  those  that  apply 
to  systems  for  which  usage  is  measured  on  a  continuous  scale,  such  as  time  in  hours  or  distance 
in  miles.  For  continuous  models,  outcomes  are  usually  measured  in  terms  of  an  interval  or 
range;  for  example,  mean  time/miles  between  failures.  Discrete  models  are  those  that  apply  to 
systems  for  which  usage  is  measured  on  an  enumerative  or  classificatory  basis,  such  as  pass/ fail 
or  go/no-go.  For  discrete  models,  outcomes  are  recorded  in  terms  of  distinct,  countable  events 
that  give  rise  to  probability  estimates. 

3.1.4  Model  Substitution. 

List  of  Notation 

Discrete  Parameters: 

N  number  of  trials  =  sample  size 

S  success 

F  failure 

NS  number  of  successes 

NF  number  of  failures 

U  unreliability 

R  reliability 


Continuous  Parameters: 

MTTF  mean  time/trials  to  failure 

MTBF  mean  time/trials  between  failures 

In  general,  continuous  models  are  designed  for  continuous  data,  and  discrete  models  are 
designed  for  discrete  data.  In  the  event  a  designated  model  is  unavailable  for  use,  it  may  be 
possible  to  use  a  continuous  model  for  discrete  data  or  a  discrete  model  for  continuous  data.  The 
latter  case  is  generally  not  a  practical  option,  though.  (The  AMSAA  Subsystem  Tracking  Model, 
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for  example,  is  a  continuous  model  that  may  be  used  with  discrete  data,  subject  to  the  conditions 
mentioned  at  the^nd  of  this  paragraph.)  In  cases  involving  model  subshtution,  the  “substitute” 
model  is  used  as  an  approximation  for  the  intended  model,  and  the  original  data  appropriate  for 
the  intended  model  must  be  converted  to  a  format  appropriate  for  the  substitute  model.  Note  that 
in  applying  a  continuous  model  to  discrete  data,  the  results  of  the  approximation  improve  as  the 
number  of  trials  increases  and  the  probability  of  failure  decreases. 


By  way  of  an  example,  we  show  a  method  for  converting  discrete  data  to  a  continuous 
format  and  vice  versa.  Suppose  that  from  a  sample  size  of  N  =  5  trials  the  following  outcomes 
are  observed,  where  S  denotes  a  success  and  F  denotes  a  failure: 

S  S  S  S  F 

The  number  of  successes,  NS,  is  four;  the  number  of  failures,  NF,  is  one;  and  N  =  NS +  NF. 
To  begin,  note  that  in  discrete  terms: 


U  =  probabilityifailure)  =  (1) 

The  reciprocal  of  U,  namely  N/NF,  may  be  viewed  as  a  measure  of  the  number  of  trials  to  the 
number  of  failures,  MTTF,  thus  allowing  a  continuous  measure  to  be  related  to  a  discrete 
measure: 


MTTF  =  ^ 

In  the  example,  MTTF  =  5  and  MTBF  =  4,  so  that: 

MTBF  =  MTTF  - 1 

Substituting  (2)  into  (3)  and  noting  that  R  =  l  —  U  results  in: 

1  ,  ^ 

MTBF  =  — -1  =  - — - 
U  l-B 


(2) 


(3) 


(4) 


Equation  (4)  is  used  to  convert  a  discrete  measure  to  a  continuous  measure.  To  convert  a 
continuous  measure  to  a  discrete  measure,  rearrange  (4)  and  solve  for  R: 

MTBF  +  1  = 


MTBF  + 1 


(6) 
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3.2 


System  Level  Reliability  Growth  Tracking  Models. 


3.2.1  Continuous  Tracking  Models. 

3.2.1.1  Background  and  Basis  tor  the  AMSAA  Continuous  Tracking  Model. 


List  of  Notation 

h 

K 

Fi 

Oi 

f 

e 

t 

F(t) 

p{t) 

A 

P 

m(t) 

T 

F 

A 

L 

U 

Y 


a 


cumulative  test  time  when  design  modification  i  is  made 
final  entry  in  a  sequence  of  test  times;  point  where  the 
last  design  modification  is  made 
constant  failure  rate  during  i-th  time  interval 

number  of  failures  during  i-th  time  interval 
mean  value  function  for  F-, 
a  particular  realization  of 
exponential  function 
cumulative  test  time 

total  number  of  system  failures  by  time  t 

mean  value  function  for  F(t) 

failure  rate  for  configuration  i  where  y 

instantaneous  system  failure  rate  at  time  t;  also  referred 

to  as  the  failure  intensity  function  /  \  /  q  m 

scale  parameter  of  parametric  function  p{t} [A  >V) 

shape  parameter  of  parametric  function  p{t) 

instantaneous  mean  time  betw.een  failures  at  time  t 

total  test  time  .  •  x 

total  observed  number  of  failures  by  time 

cumulative  time  to  i-th  failure 

denotes  an  estimate  when  placed  over  a  parameter 

lower  confidence  coefficient 

upper  confidence  coefficient 

desired  confidence  level 

denotes  an  unbiased  estimate  when  placed  over  a 

parameter 

significance  level 


The  AMSAA  Continuous  Reliability  Growth  Tracking  Model  may  be  used  to  track  the 


and  the  system  demonstrates  htgh 

reliability  during  test. 


one- 
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The  model  is  designed  for  tracking  system  reliability  within  a  test  phase  and  not  across 
test  phases.  Accordingly,  the  basis  of  the  model  is  described  in  the  following  way.  Let  the  st^ 
of  a  test  phase  be  initialized  at  time  zero,  and  let  0  =  to  <  <  ^2  <•  •  •  <  denote  the  cumu 

test  times  on  the  system  when  design  modifications  are  made.  Assume  the  system  failure  rate  is 
constant  between  successive  ,  and  let  A,  denote  the  constant  failure  rate  dunng  the  i-th  time 

interval  [t-  , The  time  intervals  do  not  have  to  be  equal  in  length.  Based  on  the  constant 
failure  rate’ 'assumption,  the  number  of  failures  F,  during  the  i-th  time  interval  is  Poisson 

distributed  with  mean  (9,.  =  A.  (t,-  - 1,_,) . 

That  is. 


Prob(/^.=/)  = 


(/  =  0,1,2,...) 


During  developmental  testing  programs,  if  more  than  one  system  prototype  is  tested  and  if  t 
proto^ies  have  the  same  basic  configuration  between  modifications,  then  under  the  constant 
failure  rate  assumption,  the  following  are  true: 

•  the  time  t-  may  be  considered  as  the  cumulative  test  time  to  the  i-th  modification,  and 

.  F.  may  be  considered  as  the  cumulative  total  number  of  failures  experienced  by  all 
system  prototypes  during  the  i-th  time  interval  [t,_i  ,t,  )  • 

The  previous  discussion  is  summarized  graphically: 
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Phase  1  I 

I 


Phase  2 


I  Phases 


Figure  1.  Failure  Rates  Between  Modifications. 

Let  t  denote  the  cumulative  test  time,  and  let  F(t)  be  the  total  number  of  system  failures 
by  time  t.  If  t  is  in  the  first  time  interval: 


0  t  t,  h 

Figure  2.  Time  Line  for  Phase  2  (t  in  first  time  interval). 

then  F(t)  has  the  Poisson  distribution  with  mean  /i,  t .  Now  if  t  is  m  the  second  time  interval. 


*4 


Figure  3.  Time  Line  for  Phase  2  (t  in  second  time  interval). 

then  F(t)  is  the  number  of  system  failures  F,  in  the  first  time  interval  plus  the  number  of  system 
failures  in  the  second  time  interval  between  r,  and  t.  The  failure  rate  for  the  first  time  interval  ,s 
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,  and  the  failure  rate  for  the  second  time  interval  is  .  Therefore,  the  mean  of  F(t)  is  the  sum 
of  the  mean  of  F^=A^  plus  the  mean  number  of  failures  from  t^  to  t,  which  is  (t  - )  •  That 
is,  F(t)  has  mean  ^t)  =  A^t^+A2{t-t^). 

When  the  failure  rate  is  constant  (homogeneous)  over  a  test  interval,  then  F(t)  is  said  to 
follow  a  homogeneous  Poisson  process  with  mean  number  of  failures  of  the  form  At .  When  the 
failure  rates  change  with  time,  e.g.,  from  interval  1  to  interval  2,  then  under  certain  conditions, 
F(t)  is  said  to  follow  a  nonhomogeneous  Poisson  process  (NHPP).  In  the  presence  of  reliability 
growth,  F(t)  would  follow  a  NHPP  with  mean  value  function: 

t 

0it)  =  \p{y)dy  (8) 

0 

where  p{y)  =  Ai,y  ,t,)  •  From  (7),  for  any  t  >  0, 

Prob[f(0  =  /]  =  (/  =  0,U,...)  (9) 

The  integer- valued  process  {F’(t),r  >  0|  may  be  regarded  as  a  NHPP  with  intensity 
function  The  physical  interpretation  of  p{t)  is  that  for  infinitesimally  small  At,  /?(?)  At 

is  approximately  the  probability  of  a  system  failure  in  the  time  interval  (t,  t  +  At);  that  is,  it  is 
approximately  the  instantaneous  system  failure  rate.  If  /7(t )  =  /I ,  a  constant  failure  rate  for  all  t, 
then  a  system  is  experiencing  no  growth  over  time,  corresponding  to  the  exponential  case.  If 
p{t)  is  decreasing  with  time,  [A^  >  . . .) ,  then  a  system  is  experiencing  reliability  growth. 

Finally,  pit)  increasing  over  time  indicates  deterioration  in  system  reliability. 

Based  on  the  learning  curve  approach,  which  is  outlined  in  detail  in  the  section  on  the 
AMSAA  Discrete  Reliability  Growth  Tracking  Model,  the  AMSAA  Continuous  Reliability 
Growth  Tracking  Model  assumes  that  p{t)  may  be  approximated  by  a  continuous,  parametric 
function.  Using  a  result  established  for  the  Discrete  Model: 

£[F(0]  =  At  ^  (10) 

and  the  instantaneous  system  failure  rate  p{t)  is  the  change  per  unit  time  of  E[F(t)]: 

p(t)  =  —E[F{t)]  =  A/^t^-'  {A,/^,t>0)  (11) 

dt 

With  a  failure  rate  pit)  that  may  change  with  test  time,  the  NHPP  provides  a  basis  for 
describing  the  reliability  growth  process  within  a  test  phase. 
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Failure 


Phase  1  '  Phase  2  '  Phase  3 

I  I 

Figure  4.  Parametric  Approximation  to  Failure  Rates  Between  Modifications. 

3.2.1.2  The  AMSAA  Continuous  Reliability  Growth  Tracking  Model.  The  AMSAA 
Continuous  Reliability  Growth  Tracking  Model  assumes  that  within  a  test  phase  failures  are 
occurring  according  to  a  nonhomogeneous  Poisson  process  with  failure  rate  (intensity  of  failures) 
represented  by  the  parametric  function: 

p(t)  =  U,A^>0)  (12) 

where  the  parameter  A  is  referred  to  as  the  scale  parameter  because  it  depends  upon  the  unit  of 
measurement  chosen  for  t,  the  parameter  P  is  referred  to  as  the  growth  or  shape  parameter 
because  it  characterizes  the  shape  of  the  graph  of  the  intensity  function  (Equation  (12)  and 
Figure  4),  and  t  is  the  cumulative  test  time.  Under  this  model  the  function: 

md)  =  4;  = 

is  interpreted  as  the  instantaneous  mean  time  between  failures  (MTBF)  of  the  system  at  time  t. 
When  t  corresponds  to  the  total  cumulative  time  for  the  system;  that  is,  t=T,  then  m(T)  is  the 
demonstrated  MTBF  of  the  system  in  its  present  configuration  at  the  end  of  test. 
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MTBF 


Figure  5.  Test  Phase  Reliability  Growth  Based  on 
AMSAA  Continuous  Tracking  Model. 

Note  that  the  theoretical  curve  is  imdefined  at  the  origin.  Typically  the  MTBF  during  the  initial 
test  interval  [0,f,]  is  characterized  by  a  constant  reliability  with  growth  occurring  beyond  t^ . 

Cumulative  Number  of  Failures 

The  total  number  of  failures  F(t)  accumulated  on  all  test  items  in  cumulative  test  time  t  is 
a  Poisson  random  variable,  and  the  probability  that  exactly  /  failures  occur  between  the  initiation 
of  testing  and  the  cumulative  test  time  t  is; 

Prob[F(t)  =  f]  =  -  (14) 

in  which  0{t)  is  the  mean  value  function;  that  is,  the  expected  number  of  failures  expressed  as  a 
function  of  test  time.  To  describe  the  reliability  growth  process,  the  cumulative  number  of 
failures  is  a  function  of  the  form  0{t)  =  Xt^ ,  where  A  and  /?  are  positive  parameters. 

Number  of  Failures  in  an  Interval 

The  number  of  failures  occurring  in  the  interval  from  test  time  /,  until  test  time  /,  >  where 
>t^  is  a  Poisson  random  variable  with  mean: 

<'5) 

According  to  the  model  assumption,  the  number  of  failures  that  occur  in  any  time  interval  is 
statistically  independent  of  the  number  of  failures  that  occur  in  any  interval  which  does  not 
overlap  the  first  interval,  and  only  one  failure  can  occur  at  any  instant  of  time. 
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Intensity  Function 

The  intensity  function  in  (12)  is  sometimes  referred  to  as  a  failure  rate;  it  is  not  the  failure 
rate  of  a  life  distribution,  rather  it  is  the  failure  rate  of  a  process,  namely  a  NHPP. 

Option  For  Individual  Failure  Time  Data 

Estimation  Procedures  For  Model 

Modeling  reliability  growth  as  a  nonhomogeneous  Poisson  process  permits  an  assessment 
of  the  demonstrated  reliability  by  statistical  procedures.  The  method  of  maximum  likelihood 
provides  estimates  for  the  scale  parameter  X  and  the  shape  parameter  p,  which  are  used  in  the 

estimation  of  the  intensity  function  /^(t)  in  (12).  In  accordance  with  (13),  the  reciprocal  of  the 
current  value  of  the  intensity  function  is  the  instantaneous  mean  time  between  failures  (MTBF) 
for  the  system.  Procedures  for  point  estimation  and  interval  estimation  for  the  system  MTBF  are 
described  in  more  detail.  A  goodness-of-fit  test  to  determine  model  suitability  is  also  described. 

The  procedures  outlined  in  this  section  are  used  to  analyze  data  for  which  (a)  the  exact 
times  of  failure  are  known  and  (b)  testing  is  conducted  on  a  time  terminated  basis  or  the  tests  are 
in  progress  with  data  available  through  some  time.  The  required  data  consist  of  the  cumulative 
test  time  on  all  systems  at  the  occurrence  of  each  failure  as  well  as  the  accumulated  total  test 
time  T.  To  calculate  the  cumulative  test  time  of  a  failure  occurrence,  it  is  necessary  to  sum  the 
test  time  on  every  system  at  the  point  of  failure.  The  data  then  consist  of  the  F  successive  failure 
times  Jif,  <  JSf2  <  ^3  <■■■<  occur  prior  to  T.  This  case  is  referred  to  as  the  Option  for 

Individual  Failure  Time  Data. 

Point  Estimation 

The  method  of  maximum  likelihood  provides  point  estimates  for  the  parameters  of  the 
failure  intensity  function  (12).  The  maximum  likelihood  estimate  (mle)  for  the  shape  parameter 
P  is: 

P  =  - -  (16) 

FlnT-j^lnA. 

/=1 

By  equating  the  observed  number  of  failures  by  time  T  (namely  F)  with  the  expected  number  of 
failures  by  time  T  (namely  E[F(T)])  and  by  substituting  mle’s  in  place  of  the  true,  but  unknown, 
parameters  in  (10)  we  obtain: 


F  =  iT^  (17) 

from  which  we  obtain  an  estimate  for  the  scale  parameter  X : 

i  =  4-  (18) 

r^P 
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For  any  time  t  >  0,  the  failure  intensity  function  is  estimated  by: 

>(t)  -  XJn.*-' 

In  particular.  (19)  holds  for  the  total  test  time  T.  By  substitution  from  (17).  the  estimator  XT) 
can  be  written  as: 


^(T)  =  /i/?T 


P 


X 


=  P 


F 

T 


(20) 


where  F/T  is  the  estimate  of  the  intensity  function  for  a  homogeneous  Poisson  process  Hence 
the  fraction  (l  -  of  the  initial  failure  intensity  is  effectively  removed  by  time  T,  resulting  m 

(20). 

Finally,  the  reciprocal  of  p{T)  provides  an  estimate  of  the  mean  time  between  failures  of 
the  system  at  the  time  T  and  represents  the  system  reliability  growth  under  the  model; 

.h(T)  =  ^  (2» 

/?(T) 


“'"“‘inteml  esLates  provide  a  measure  of  the  uncertainty  regarding  a  parameter^  For  the 
reliability  growth  process,  the  parameter  of  primary  interest  is  the  system  mean  time  between 
fa  lures  at  Se  end  of  test.  m(T).  The  probability  distribution  of  the  point  estimate  for  the 
[nlenrity  function  at  T,  MT)  .  ^  the  basis  for  the  interval  estimate  for  the  true  (but  unknown) 

value  of  the  intensity  function  at  T .  piT ) . 

These  interval  estimates  are  referred  to  as  confidence  intervals  and  may  be  computed  for 
selected  confidence  levels.  The  values  in  Table  1  facilitate  computation  of 
intervals  for  m(T)  by  providing  confidence  coefficients  L  and  U  correyonding  o  e 
bound  and  upper  boimd,  respectively.  These  coefficients  are  indexed  by  the  total  number  of 
^eld  Jmes  F  and  the  Lsired  confidence  level  y.  The  two-sided  confidence  interval  for 

m(T)  is  thus: 


Lp.m(T)  <  m(T)  <  Up,>(T) 


(22) 


Table  2  may  be  used  to  compute  one-sided  interval  estimates  (lower  confidence  bounds)  for 
m(T)  such  that: 

Lp,,m(T)  <  m(T) 
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Note  that  both  tables  are  to  be  used  only  for  time  terminated  growth  tests.  Also,  since  the 
number  of  failures  has  a  discrete  probability  distribution,  the  interval  estimates  in  (22)  and  (23) 
are  conservative;  that  is,  the  actual  confidence  level  is  slightly  larger  than  the  desired  confidence 
level  y. 
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TABLE  1.  LOWER  (L)  AND  UPPER  (U)  COEFFICIENTS 
FOR  CONFIDENCE  INTERVALS  FOR  MTBF  FROM 
TIME  TERMINATED  RELIABILITY  GROWTH  TEST 
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For  F>  100,  L 


in  which 


and  U 


is  the  lOOx  (^.5  +  ^)  -  th  percentile  of  the  standard  normal  distribution. 
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Goodne^^omt^^^  where  the  individual  failure  times  are  known,  a  Cramer-von  Mises  st^istic  is 
used  to  test  the  null  hypothesis  that  a  nonhomogeneous  Poisson  process  with  failure  intensity 
faS  2)  properly  describes  the  reliability  growth  of  a  system.  To  calculate  the  statistic,  an 

unbiased  estimate  of  the  shape  parameter  ft  is  used. 


(24) 


This  unbiased  estimate  of  /}  is  for  a  time  terminated  reliability  growth  test  with  F  observed 
failures.  The  goodness-of-fit  statistic  is. 


Cf 


/  2/-1 

T 

J  2F 

(25) 


where  the  failure  times  X,  must  be  ordered  so  that  0  <  X,  <  <  ...  <  X^. 

The  null  hypothesis  that  the  model  represents  the  observed  data  is  rejected  if  the  statistic 
C  exceeds  the  critical  value  for  a  chosen  significance  level  a.  Cntical  values  of  C,  or 

.20,  .15,  .10,  .05,  .01  are  shown  in  Table  3  where  the  table  is  indexed  by  F,  the  total 

number  of  observed  failures. 
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TABLE  3. 


critical  values  for  cramer-von  mises  goodness-of 

CRITICAL  failure  TIME  DATA 


^FIT  TEST 


For  F  >  100  use  values  for  F  100. 
a  =  significance  level 

Besides  using  statistical  methods  for  (“n  ?n 

Fig^S  lata,  provide  a  graphic  description  of  test  results 

and  should  always  be  part  of  the  reliability  analysis. 

I  rirumon^trates  the  option  for  individual  failure  time  data  in  which 
The  following  example  demonstrates  P  incorporation  of  design  changes. 

two  prototypes  of  a  system  are  tested  concu^  y  ^  one  of  the  gSwth  subsystems  in  the  example 

(The  data  in  this  example  LtRACK  )  The  first  prototype  is  tested  for 

for  the  AMSAA  Subsystem  Tracking  Model  -  SS  of  T  =  300  cumulative  test  hours. 
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model,  such  data  can  result  from  rounding  and  are  computationally  tractable  using  the  statistical 
estimation  procedures  described  previously  for  the  model.  Note  that  the  data  are  from  a  time 
terminated  test. 

TABLE  4.  TEST  DATA  FOR  INDIVIDUAL  FAILURE  TIME  OPTION 

(An  asterisk  denotes  the  failed  system.) 


Failure 

Number 

Prot.  #1 
Hours 

Prot.  #2 
Hours 

Cum 

Hours 

Failure 

Number 

Prot.  #1 
Hours 

Prot.  #2 
Hours 

Cumulative 

Hours 

1 

2.6* 

.0 

2.6 

15 

60.5 

37.6* 

98.1 

2 

16.5* 

.0 

16.5 

16 

61.9* 

39.1 

101.1 

3 

16.5* 

.0 

16.5 

17 

76.6* 

55.4 

132.0 

4 

17.0* 

.0 

17.0 

18 

81.1 

61.1* 

142.2 

5 

20.5 

21.4 

19 

84.1* 

63.6 

147.7 

6 

25.3 

3.8* 

29.1 

20 

84.7* 

64.3 

149.0 

7 

28.7 

4.6* 

33.3 

21 

94.6* 

72.6 

167.2 

8 

41.8* 

14.7 

56.5 

22 

104.8 

85.9* 

,  190.7 

9 

45.5* 

17.6 

63.1 

23 

105.9 

87.1* 

193.0 

10 

48.6 

22.0* 

70.6 

24 

108.8* 

89.9 

198.7 

11 

49.6 

23.4* 

73.0 

25 

132.4 

119.5* 

251.9 

12 

51.4* 

26.3 

77.7 

26 

132.4 

150.1* 

282.5 

13 

58.2* 

35.7 

93.9 

27 

132.4 

153.7* 

286.1 

14 

59.0 

36.5* 

95.5 

End 

132.4 

167.6 

300.0 

By  using  the  27  failure  times  listed  under  the  columns  labeled  “Cumulative  Hours”  in 
Table  4  and  by  applying  (16),  (18),  (19)  and  (21)  we  obtain  the  following  estimates.  The  point 

estimate  for  the  shape  parameter  is  j3=0.1\6\  the  point  estimate  for  the  scale  parameter  is 
X  ^  0.454 ;  the  estimated  failure  intensity  at  the  end  of  the  test  is  ^(T)  =  0.0645  failures  per 
hour;  the  estimated  MTBF  at  the  end  of  the  300-hour  test  is  m  (T)  =  15.5  hours.  As  shown  in 
Figure  6,  superimposing  a  graph  of  the  estimated  intensity  function  (19)  atop  a  plot  of  the 
average  failure  rate  (using  six  50-hour  intervals)  reveals  a  decreasing  failure  intensity  indicative 
of  reliability  growth. 
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Figured  Estimated  Intensity  Function  Superimposed  On  Average 

Failure  Rate  Plot  From  Observed  Data. 


Using  (22),  Table  1  and  a  confidence  level  of  90  percent,  the  two-sided  interval  estimate 
for  the  MTBF  at  the  end  of  the  test  is  [9.9, 26.1].  These  results  and  the  estimated  MTBF 
tracking  growth  curve  (substituting  t  for  T  in  (21))  are  shown  in  Figure  7. 
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30 


26.1  - 


Figure  7.  Estimated  MTBF  Function  With  90  Percent 
Interval  Estimate  at  T  =  300  Hours. 


Finally,  to  test  the  model  goodness-of-fit,  a  Cramer-von  Mises  statistic  is  compared  to  the 
critical  value  from  Table  3  corresponding  to  a  chosen  significance  level  a  -  0.05  and  tota 
observed  number  of  failures  F  =  27.  Linear  interpolation  is  used  to  amve  at  the  cntical  value^ 
Since  the  statistic,  0.091,  is  less  than  the  critical  value,  0.218.  we  accept  the  hypothesis  that  the 
AMS  AA  Continuous  Reliability  Growth  Tracking  Model  is  appropnate  for  this  data  set. 


Option  for  Grouped  Data 


List  of  Notation 

K 

i 

ti 

Fi 

A 

P 


number  of  intervals  (or  groups)  or  the  last  group 
interval  number 

time  at  beginning  (or  end)  of  interval 
observed  number  of  failures  in  interval  ) 
total  test  time 

denotes  an  estimate  when  placed  over  a  parameter 
shape  parameter  (/?  >  O) 
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A 

scale  parameter  {A  >  O) 

/<t) 

instantaneous  failure  intensity  at  time  t 

m{t) 

instantaneous  MTBF  at  time  t 

Mk 

MTBF  for  the  last  group 

Ek 

expected  number  of  failures  in  the  last  group 

Pk 

failure  intensity  for  the  last  group 

F 

total  observed  number  of  failures 

L 

lower  confidence  coefficient 

U 

upper  confidence  coefficient 

Y 

specified  confidence  level 

Ei 

expected  number  of  failures  in  interval  i 

Kr 

number  of  intervals  after  recombination  of  intervals 

Oi 

observed  number  of  failures  in  interval  i 

x" 

chi-squared  value 

Reliability  growth  parameters  can  be  estimated  in  accordance  with  the  AMSAA 
Continuous  Tracking  Model  even  if  the  exact  times  of  failure  are  unknown  and  all  that  is  known 
is  the  number  of  failures  that  occurred  in  each  interval  of  time,  provided  there  are  at  least  three 
intervals  and  at  least  two  intervals  have  failures.  This  case  is  referred  to  as  the  Option  for 
Grouped  Data.  This  section  describes  the  estimation  procedures  and  goodness-of-fit  procedures 
for  analyzing  such  data  and  provides  an  example  of  model  usage.  In  the  following  discussion, 
the  words  “group”  and  “interval”  are  interchangeable. 

Estimation  Procedures  for  Model 

The  required  data  consist  of  the  total  number  of  failures  in  each  of  K  intervals  ot  test 
time.  The  first  interval  always  starts  at  test  time  zero  so  that  to  =  0 .  The  groups  do  not  have  to 
be  of  equal  length.  The  observed  number  of  failures  in  the  interval  from  t,.,  to  is  denoted  by 

Pi- 

Point  Estimation 

The  method  of  maximum  likelihood  provides  point  estimates  for  the  parameters  oi  the 
model.  The  maximum  likelihood  estimate  for  the  shape  parameter  /?  is  the  value  that  satisfies 

the  following  nonlinear  equation: 


fflnf,-/,^ilnt,._, 


tf-tt 


(26) 


in  which  to  In  to  is  defined  as  zero. 
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By  equating  the  total  expected  number  of  failures  to  the  total  observed  number  of  failures. 

H  -  h.  (27) 

i  =  l 

and  solving  for  A ,  we  obtain  an  estimate  for  the  scale  parameter: 

K 

ZFi 

i  =  (28) 

‘'K 

i 

Point  estimates  for  the  intensity  function  pH)  and  the  mean  time  between  failures 
function  m{t)  are  calculated  as  in  the  previous  section  that  describes  the  Option  for  Individual 
Failure  Time  Data;  that  is, 


^t)  -  (i,Ai>o)  (29) 

m(t)  =  (30) 

The  functions  in  (29)  and  (30)  provide  instantaneous  estimates  that  give  rise  to  smooth 
continuous  curves,  but  these  functions  do  not  describe  the  reliability  growth  that  occurs  on  a 
configuration  basis  representative  of  grouped  data.  Under  the  model  option  for  grouped  data,  the 
estimate  for  the  MTBF  for  the  last  group,  Mk  ,  is  the  amount  of  test  time  in  the  last  group 
divided  by  the  estimated  expected  number  of  failures  in  the  last  group: 

M  =  (31) 

where  the  estimated  expected  number  of  failures  in  the  last  group  is: 

E,  =  (32) 

From  (3 1 )  we  obtain  an  estimate  for  the  failure  intensity  for  the  last  group: 


Interval  Estimation 

Approximate  lower  confidence  bounds  and  two-sided  confidence  intervals  may  be 
computed  for  the  MTBF  for  the  last  group.  Using  (31)  and  Table  1,  a  two-sided  approximate 
confidence  interval  for  may  be  calculated  from: 
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(34) 


<  Up.^K 

and  using  (31)  and  Table  2,  a  one-sided  approximate  interval  estimate  for  may  be  calculated 
from: 


Lp.^M^  <  (35) 

where  F  is  the  total  observed  number  of  failures  and  y  is  the  desired  confidence  level. 
Goodness-of-Fit 

A  chi-squared  goodness-of-fit  test  is  used  to  test  the  null  hypothesis  that  the  AMS  AA 
Continuous  Reliability  Growth  Tracking  Model  adequately  represents  a  set  of  grouped  data.  The 
expected  number  of  failures  in  the  interval  from  to  t,-  is  approximated  by: 

E,  =  (36) 

Adjacent  intervals  may  have  to  be  combined  so  that  the  estimated  expected  number  of 
failures  in  any  combined  interval  is  at  least  five.  Let  the  number  of  intervals  after  this 
recombination  be  Kj^ ,  and  let  the  observed  number  of  failures  in  the  i-th  new  interval  be  and 

A 

the  estimated  expected  number  of  failures  in  the  i-th  new  interval  be  .  Then  the  statistic: 

,-A 
A 

is  approximately  distributed  as  a  chi-squared  random  variable  with  —  2  degrees  of  freedom. 
The  null  hypothesis  is  rejected  if  the  ^  statistic  exceeds  the  critical  value  for  a  chosen 
significance  level.  Critical  values  for  this  statistic  can  be  found  in  tables  of  the  chi-squared 
distribution. 

Besides  using  statistical  methods  for  assessing  model  goodness-of-fit,  one  should  also 
construct  an  average  failure  rate  plot  or  a  superimposed  expected  failure  rate  plot  (as  shown  in 
Figure  6).  Derived  from  the  failure  data,  these  plots  provide  a  graphic  description  of  test  results 
and  should  always  be  part  of  the  reliability  analysis. 

Example 

The  following  example  uses  aircraft  data  to  demonstrate  the  option  for  grouped  data. 

(The  data  in  this  example  are  used  subsequently  for  one  of  the  growth  subsystems  in  the  example 
for  the  AMSAA  Subsystem  Tracking  Model  -  SSTRACK.)  In  this  example,  an  aircraft  has 
scheduled  inspections  at  intervals  of  twenty  flight  hours.  For  the  first  100  hours  of  flight  testing 
the  results  are: 


(37) 
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TABLE  5.  TEST  DATA  FOR  GROUPED  DATA  OPTION 


Start  Time 

End  Time 

Observed  Number  of  Failures 

13 

20 

40 

16 

5 

60 

80 

8 

80 

100 

7 

There  are  a  total  of  F  =  49  observed  failures  from  K  =  5  intervals.  Solution  of  (26)  for  P  yields 
an  estimate  of  0.753  for  the  shape  parameter.  From  (28)  the  scale  parameter  estimate  is  1 .53. 
For  the  last  group,  the  intensity  function  estimate  is  0.379  failures  per  flight  hour  and  the  MTBF 
estimate  is  2.6  flight  hours.  Table  6  shows  that  those  adjacent  intervals  do  not  have  to  be 
combined  after  applying  (36)  to  the  original  intervals.  Therefore,  =  5 . 

TABLE  6.  OBSERVED  VERSUS  ESTIMATE  OF  EXPECTED 
NUMBER  OF  FAILURES  FOR  TEST  DATA  FOR  GROUPED  DATA  OPTION 


Start  Time 

End  Time 

Observed  Number  of 
Failures 

Estimated  Expected 
Number  of  Failures 

0 

20 

13 

14.59 

20 

40 

16 

9.99 

40 

60 

5 

8.77 

60 

80 

8 

8.07 

80 

100 

7 

7.58 

To  test  the  model  goodness-of-fit,  a  chi-squared  statistic  of  5.5  is  compared  to  the  critipal 
value  of  7.8  corresponding  to  3  degrees  of  freedom  and  a  0.05  significance  level.  Since  the 
statistic  is  less  than  the  critical  value,  the  applicability  of  the  model  is  accepted. 

3.2.2  AMSAA  Discrete  Tracking  Model. 

3.2.2.1  Background  and  Motivation  for  Model. 

List  of  Notation 

t  cumulative  test  time 

K(t)  cumulative  number  of  failures  by  time  t 
c(t)  cumulative  failure  rate  by  time  t 
In  natural  logarithm  function  (base  e) 

5  constant  term  representing  the  y-intercept  of  a  linear  equation 

a  constant  term  representing  the  slope  of  a  linear  equation 

X,  scale  parameter  ( i  >  0 )  of  power  function 
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shape  parameter  ( /?  >  0 )  of  power  flmction;  ^  -  1  « 

cumutXTnmto  of  trials  through  configuration  i 
summation  of 

number  of  trials  in  configuration  i 
cumulative  number  of  failures  through  configuration  i 
number  of  failures  in  configuration  i 

irj.  > 

e[k  ]  expected  value  of 

'  probability  of  failure  for  configuration  i 

g.  probability  of  failure  for  trial  1 

reliability  for  configuration  i  (or  trial  i) 
denotes  an  estimate  when  placed  over  a  parameter 


r, 

s 

M 


R 


^  cenoicb  <ui  -  —  JT 

reliability  growth  trachingnte^^^^^ 

that  is  consistent  ^^^el  development  and  maximum  likelihood  estimation 

^^eS  development. 

The  motivation  for  the  AMSAA  ~ 
the  learning  curve  approach  for  continuous  data  as  follows. 

Let  t  denote  the  cumulative  test  time,  and  let  K(0  denote  the  cumulative  number  of 
failures  by  tiS  The  cumulative  failure  rate,  c(t),  is  the  ratio: 


c(t)  = 


K(t) 

t 


(38) 


the  logarithm  of  the  cumulative  test  time; 


Inc(t)  =  ^-^3rlnt 


(39) 


failure  rate  becomes: 


c(  t )  -  /it" 


(40) 


By  substitution  from  (38), 


K(0  .  ir- 

t 


(41) 
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Multiplying  both  sides  of  (41)  by  t  and  letting  /?=  1  -  « ,  the  cumulative  number  of  failures  by  t 
becomes: 

K{t)  =  it^ 

This  power  function  oft  is  the  learning  curve  property  for  K(t),  where  ^  0 . 

3  2  2  2  Model  Development.  To  construct  the  AMSAA  Discrete  Reliability  Growth 

XX  ^ottrrfXe®'^raS“tm^^^^^^  Sraphically  as 

a  sequence  of  decreasing,  horizontal  steps. 

We  then  note  the  special  case  where  the  configuration  size  is  one  for  all 
graphically  as  a  decreasing,  smooth  curve. 

c-=35H£.SS=Fi™“ 

number  of  trials  during  configuration  i,  and  let  M ,  be  the  number  of  farlures  during 
configuration  i.  Then  the  cumulative  number  of  trials  through  configuration namely  T. , 

sum  of  the  N ;  for  all  i*. 


T,  =  XN, 


(43) 


and  the  cumulative 
for  all  i: 


ive  number  of  failures  through  configuration  i,  namely  K, ,  is  the  sum  of  the  M , 

■c’..  (44) 

K,  " 

We  express  the  expected  value  of  K,  as  E[K,1  and  define  it  as  the  expected  number  of 
failures  by  the  end  of  configuration  i.  Applying  the  learning  curve  property  to  E^K,  ]  rmplies. 

E[Kil  =  iT," 

We  introduce  a  term  for  the  probability  of  failure  for  configuration  one,  n^ely  f, ,  and 
use  it  to  develop  a  generalized  equation  for  f,  in  terms  of  the  T,  and  N,  From  (45),  the 
expected  number  of  failures  by  the  end  of  configuration  one  is: 
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E[K,]  =  ATf  =  f,N, 


f,  = 


AT/ 

N. 


(46) 


of  failures  in  configuration  two,  we  obtain; 

AT^ -AT^ 

eIkJ  =  =  f,N, +f2N2  =  iT,^+f2N2 


f,  = 


N, 


(47) 


By  this  method  of  inductive  reasoning  we  obtain  a  generalized  equation  for  the  failure 
probability,  f; ,  on  a  configuration  basis: 


f  = 


ATf-ATf., 


(48) 


N; 


and  use  (48)  for  the  grouped  data  option. 

For  the  special  case  where  N,  =  1  for  all  1,  (48)  becomes  a  smooth  curve,  g, .  that 
represents  the  probability  of  failure  for  the  option  for  trial  by  trial  data; 


g.  =  Ai^-A(i-iy 


(49) 


(51) 


In  (49)  i  represents  the  trial  number.  Note  that  T,  =  0 ,  so  that  (48)  reduces  to  (46)  when  i  - 1 . 
AiL.  for  i  =  1  in  (49),  g,  =  i .  Using  (48)  we  obtain  an  equation  for  the  reliability  (proba  1 1  y 

of  success)  for  the  i-th  configuration: 

=  1-/,  , 

and  using  (49)  we  obtain  an  equation  for  the  reliability  for  the  i-th  tnal: 

i?,  =  1-^1 

/Ao\  anH  \  are  the  exact  model  equations  for  tracking  the  reliability 

^71^  Fstimation  Procedures.  This  section  describes  procedures  for  estinriating  the 
V  thp  AMSAA  Discrete  Reliability  Growth  Tracking  Model.  It  also  includes  an 
^p^^^xtXn  /qllX  calculating  reliability  lower  confidence  bounds  and  an  example 

illustrating  these  concepts. 

The  estimation  procedures  described  below  provide  maximum  likelihood 
(mle’s)  for  the  model’s  two  parameters,  X  and  p,  where  X  is  the  scale  parameter  an 
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shape  (or  growth)  parameter.  The  mle's  for  X  and  p  allow  for  point  estimates  for  the  probability 
of  failure: 


A 


(52) 


and  the  probability  of  success  (reliability): 

A  =  I-/: 


(53) 


for  each  configuration  i. 

Point  Estimation 

Let  i  and  be  the  mle’s  for  X  tmd  P  respectively,  i.e.  let  (i.  such  that  (X.  P) 

maxtmiaesthedlscretemodellihelihood^^^^ 

Ri  denote  the  corresponding  estimate  of  Ri.  If 

=  satisfies  the  following  likelihood  equations: 


=  0 


(54) 


and 


=  0 


(55) 


We  recommend  using  the  model  mle’s  only  for  this  case  Situations  can  occtn  when  the 
likelihood  is  m^imized  at  a  whel“«tl  onfte  fol 

KrwrsruiThtSirs^^^ 
illXCpS"™  aX:Xont*"m:dd 

Swer  cfnS::  bouSLed  on  the  total  number  of  successes  and  trials  would  provtde  a 
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conservative 


assessment  of  ttte  achieved  reliability  Ra  under  the  assumption  that  R.  2  R,  for  i 


following  data  requirements  for  using  the  model: 


From  (54)  and  (55)  we  note  the 
Data  Requirements 

K  number  of  configurations  (or  the  final  configuration) 

M..  number  of  observed  failures  for  configuration  i 

iv'  number  oftrials  for  configuration! 

r '  cumulative  number  oftrials  through  configuration . 

Interval  Estimation  confidence  bound)  for  the  reliability  of  the  final 

conC:“:Tb»btined  Lm  the  approximation  et,uation: 


(last) ' 


LCB,  = 


(56) 


where 


LCB^  = 


R. 


n 


X 


an  approximate  lower  confidence  bound  at  the  gamma  (y)  confidence 

level  for  the  reliability  of  the  last  confignration,  where  y  ts  a  decimal 

number  in  the  interval  (0,1)  v  i  + 

a  maximum  likelihood  estimate  for  the  reliability  of  the  last 

rS'Tber  of  observed  failures  (summed)  over  all 

Ae'^^r^rc»tilepdmof  the  chi-squared distrib^^^^ 

n+2  degrees  of  fireedom 


3.2.2.4  Goodness-of-Fit.  provided  there  is  . _f«^^ 

number  of  failures  per  group,  Growth  Tracking  Model  adequately  represents 

hypothesis  that  the  AMS^  Discre^  conditions  are  met,  then  one 

Reliability  Growth  Tracking  Model. 

Besides  using  statistical  methods  for  ®ed  shown  in 

«  tirdma!  E  plots  provide  a  graphic  description  of  test  results 

Ld  should  always  be  part  of  the  reliability  analysis. 
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3.2.2.5  Example.  The  following  example  is  an  application  of  the  grouped  data  option  of 
the  AMSAA  Discrete  Reliability  Growth  Tracking  Model  for  a  system  having  four 
configurations  of  development  test  data: 

TABLE  7.  TEST  DATA  FOR  GROUPED  DATA  OPTION 


Configuration 
Number,  i 

K  =  4 

Observed  Number 
of  Failures  in 
Configuration  i 

M, 

Number  of  Trials  in 
Configuration  i 

Cumulative  Number 
of  Trials  Through 
Configuration  i 

Ti 

1  1 

5 

14 

14 

2 

3 

19  1 

33 

3 

4 

15 

48 

4 

4 

20 

68 

This  is  represented  graphically  as; 

(Ml  =5)  (M2  =  3)  (M3  =  4)  (M4  =  4) 

5 - M  33  48  ^  ^ 

(N,  =  14)  (N2=19)  (N3  =  15)  (N4-2O) 

T,  T2  T3  U 

Figure  8.  Test  Data  for  Grouped  Data  Option. 

The  solution  of  (54)  and  (55)  provides  mle’s  for  and  p  corresponding  to  0.595  and 
0.780,  respectively.  Using  (52)  and  (53)  results  in  the  following  table: 


TABLE  8  ESTIMATED  FAILURE  RATE  AND  ESTIMATED  RELIABILITY  BY 

CONFIGURATION 


Configuration  Number,  i 

K  =  4 

Estimated  Failure 
Probability  for 
Configuration  i 

h 

Estimated  Reliability  for 
Configuration  i 

A 

Ri 

1 

.333 

Ml 

2  n 

.234 

.766 

3 

.206 

.794 

4 

.190 

.810 
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^  of  Failure 


A  plot  of  the  estimated  failure  rate  by  configuration  is: 


and  a  plot  of  the  estimated  reliability  by  configuration  is: 


14  33  48 


Cumulative  Number  of  Trials,  T; 

Figure  10.  Estimated  Reliability  by  Configuration. 

Finally,  (56)  is  used  to  generate  the  following  table  of  approximate  LCB’s  for  the  reliability  of 
the  last  configuration: 


TABLE  9.  TABLE  OF  APPROXIMATE  LOWER  CONFIDENCE  BOUNDS 
(LCB’S)  FOR  FINAL  CONFIGURATION 


3.3  Subsystem  Level  Reliability  Growth  Tracking  Models. 

3.3.1  AMSAA  SSTRACK  Model  Description  and  Conditions  For  Usage.  The 
AMSAA  Subsystem  Tracking  Model  (SSTRACK)  is  a  tool  for  assessing  system  level  reliability 
from  lower  level  test  results.  The  methodology  was  developed  to  make  greater  use  of  component 
or  subsystem  test  data  in  estimating  system  reliability.  By  representing  the  system  as  a  series  of 
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independent  subsystems,  the  methodology  permits  an  assessment  of  the  system  level 
demonstrated  reliability  at  a  given  confidence  level  from  the  subsystem  test  data.  This  system 
level  assessment  is  permissible  provided  the: 

•  subsystem  test  conditions/usage  are  in  conformance  with  the  proposed  system  level 
operational  environment  (as  embodied  in  the  Operational  Mode  Summary/Mission 
Profile  [OMS/MP])  and 

•  Failure  Defmitions/Scoring  Criteria  (FD/SC)  formulated  for  each  subsystem  are 
consistent  with  the  FD/SC  used  for  system  level  test  evaluation. 

The  SSTRACK  methodology  supports  a  mix  of  test  data  from  growth  and  non-growth 
subsystems.  Statistical  goodness-of-fit  procedures  are  used  for  assessing  model  applicability  for 
growth  subsystem  test  data.  For  non-growth  subsystems,  the  model  uses  fixed  configuration  test 
data  in  the  form  of  the  total  test  time  and  the  total  number  of  failures.  The  model  applies  the 
Lindstrdm-Madden  method  [4]  for  combining  the  test  data  from  the  individual  subsystems. 
Twenty-five  subsystems  can  be  represented  by  the  current  implementation  of  the  model. 
SSTRACK  is  a  continuous  model,  but  it  may  be  used  with  discrete  data  if  the  number  of  trials  is 
large  and  the  probability  of  failure  is  small. 

A  potential  benefit  of  this  methodology  is  that  it  may  allow  for  reduced  system  level 
testing  by  combining  lower  level  subsystem  test  results  in  such  a  manner  that  system  reliability 
may  be  demonstrated  with  confidence.  Another  potential  benefit  is  that  it  may  allow  for  an 
assessment  of  the  degree  of  subsystem  test  contribution  toward  demonstrating  a  system 
reliability  requirement.  Finally,  as  mentioned,  it  may  serve  as  an  effective  means  of  combining 
test  data  from  dissimilar  sources,  namely  growth  and  non-growth  subsystems. 

Besides  the  two  provisos  stated  in  the  opening  paragraph  regarding  OMS/MP 
conformance  and  FD/SC  consistency,  a  caveat  in  using  the  methodology  is  that  high-risk 
subsystem  interfaces  should  be  identified  and  addressed  through  joint  subsystem  testing.  Also, 
as  in  any  reliability  growth  test  program,  growth  subsystem  configuration  changes  must  be 
properly  documented  for  the  methodology  to  provide  meaningful  results. 

The  primary  output  from  the  SSTRACK  computer  implementation  is  a  table  of 
approximate  lower  confidence  bounds  for  the  system  reliability  (MTBF)  for  a  range  of 
confidence  levels. 

3.3.2  Methodology. 

LIST  OF  NOTATION 

A  denotes  an  estimate  when  placed  over  a  parameter 

M  Mean  Time  Between  Failures  (MTBF) 

D  demonstration 

G  growth 

LCB  Lower  Confidence  Bound 
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7 

T 

N 

^df.r 


gamma  =  confidence  level 

(total)  test  time 

(total)  number  of  failures 

chi-squared  percentile  point  for  df  degrees  of  freedom  and  y 
beta  =  growth  parameter  from  reliability  growth  tracking  model 


To  be  able  to  handle  a  mix  of  test  data  from  growth  and  non-growth  subsystems,  the 
methodology  convex  all  all  subsystem 


yield; 


The  “equivalent”  demonstration  data  estimators  and  the  growth  data  estimators  must 


(1)  the  same  subsystem  MTBF  point  estimate  and 

(2)  the  same  subsystem  MTBF  lower  confidence  bound. 

In  other  words,  the  methodology  maintains  the  following  relationships,  respectively: 

=  Ma 

lcb^{d)  =  lcb^{g) 


(57) 

(58) 


where 


(59) 


lcb^{d)  = 


2T^ 


(60) 


Reducing  growth  subsystem  test  data  to  “equivalent”  demonstration  test  data  using  the 
following  equations  closely  satisfies  the  relationships  cited  above. 


No 


(61) 
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(62) 


T 


ip 


The  growth  estimate  for  the  MTBF,  ,  and  the  estimate  for  the  growth  parameter,  p,  are 
described  in  the  sections  on  point  estimation  for  system  level  Continuous  Reliability  Growth 
Tracking  Models. 


The  model  then  uses  the  above  equations  to  compute  an  approximate  lower  confidence 
bound  for  the  serial  system  reliability  (MTBF)  from  f  bsystem  demonstrat^ 

and  growth  subsystem  “equivalent”  demonstration  data  as  described  m  the  following  secti 

the  Lindstrdm-Madden  method. 


3.3.3  Lindstrom-Madden  Method.  In  addition  to  using  the  notation  defined  m  the 
previous  section  on  Methodology,  subsequent  equations  use  the  following  notation: 


LIST  OF  NOTATION 


sys 

system  level 

min 

minimum  of 

K 

number  of  subsystems  in  serial  system 

p 

failure  rate 

i 

subscript  for  subsystem  number 

S 

summation  of 

To  compute  an  approximate  lower  confidence  bound  (LCB)  for  the  system  MTBF  from 

subsys,erZL.«.iorand  ••equivalen.’Memonstra.ion  data.  ,he  SST^^ 

uses  an  adaptation  of  the  Lindstrdm-Madden  method  by  computing  the  following  four  estimates. 


1.  the  equivalent  amount  of  system  level  demonstration  test  time.  (This  estimate  is  a 

reflection  of  the  least  tested  subsystem  because  it  is  the  minimum  demonstration  test 
time  of  all  the  subsystems.). 


2.  the  current  system  failure  rate,  which  is  the  sum  of  the  estimated  failure  rate  from 
each  subsystem  i,  i  =  1..K, 

3.  the  “equivalent”  number  of  system  level  demonstration  failures,  which  is  the  product 
of  the  previous  two  estimates,  and 

4.  the  approximate  LCB  for  the  system  MTBF  at  a  given  confidence  level,  which  is  a 
function  of  the  equivalent  amount  of  system  level  demonstration  test  time  and  the 
equivalent  number  of  system  level  demonstration  failures. 
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In  equation  form,  these  system  level  estimates  are,  respectively: 


where 


T 

D,sys 

=  minr^i for  i  =  \..K 

(63) 

II 

K 

^  lA 

i=l 

(64) 

Pi  = 

1 

Mo.i 

(65) 

the  current  MTBF  estimate  for  subsystem  i 

^D.sys 

Psys  ^  '^D,sys 

(66) 

LCB^ 

2T 

D,sys 

(67) 

3.3.4  Example.  The  following  example  is  an  application  of  the  AMSAA  Subsystem 
Level  Reliability  Growth  Tracking  Model  to  a  system  composed  of  three  subsystems:  one  non¬ 
growth  and  two  growth  subsystems.  Besides  showing  that  SSTRACK  can  be  used  for  test  data 
gathered  from  dissimilar  sources  (namely,  non-growth  and  growth  subsystems),  this  particular 
example  was  chosen  to  show  that  system  level  reliability  estimates  are  influenced  by  - 

•  the  least  tested  subsystem  and 

•  the  least  reliable  subsystem,  that  is,  the  subsystem  with  the  largest  failure  rate. 

Subsystem  1  in  this  example  is  a  non-growth  subsystem  consisting  of  fixed  configuration 
data  of  8,000  hours  of  test  time  and  2  observed  failures. 

Subsystem  2  is  a  growth  subsystem  with  individual  failure  time  data.  In  900  hours  of  test 
time  there  were  27  observed  failures  occurring  at  the  following  cumulative  times:  7.8, 49.5, 

49.5,  51.0, 64.2,  87.3,  99.9,  169.5, 189.3,  211.8,  219.0,  233.1,  281.7,  286.5,  294.3,  303.3,  396.0, 
426.6,  443.1,  447.0,  501.6,  572.1,  579.0,  596.1,  755.7,  847.5,  858.3. 

Subsystem  3  is  also  a  growth  subsystem  with  individual  failure  time  data.  In  400  hours 
of  test  time  there  were  16  observed  failures  occurring  at  the  following  cumulative  times:  15.04, 
25,26,  47.46,53.96,  56.42,  99.57,  100.31,  111.99,  125.48,  133.43,  192.66,  249.15,285.01, 
379.43,  388.97,  395.25. 
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The  following  table  shows  the  pertinent  statistics  for  each  subsystm  i.  Jt  here  that  all 
growth  (G)  subsystem  test  data  are  reduced  to  equivalent  demonstration  (D)  test  data. 

TABLE  10.  SUBSYSTEM  STATISTICS 


Statistics 
fi  =  1,2,3) 

Subsystem  1 
fNon-growth) 

Subsystem  2 
(Growth) 

Subsystem  3 
(Growth) 

To, 

N/A 

900 

400 

^GJ 

N/A 

27 

16 

N/A 

46.53 

31.37 

^Gi 

N  = 

2 

13.5 

8 

11 

X 

8000 

628.19  1 

250.95 

4000 

46.53 

31.37 

1 

2.50  X  lO"* 

2.149  X  10'^ 

3.188  X  10'^ 

System  level  statistics  are  computed  by  applying  the  Lindstrom-Madden  method  to  the 
equivalent  demonstration  data  from  each  subsystem. 

=  niinro,,.(,.,,,2.3)  =  251.0  (68) 


P. 


5ys 


=  =  5.362x10' 


1=1 


M 


D,$ys 


—  =  18.7 

Ks 

^D,sys  ~  ^D.sys  ^  Psys  ~ 


LCB 


.80 


_  {^^'^D.sys  )  _  1^22  {confidence  level  =^S0Vo) 


/l'2Wo.„.  +  2..80 


Finally,  a  table  of  approximate  lower  confidence  bounds  is  shown  for  the  system 
reliability  (MTBF)  for  a  range  of  confidence  levels. 


(69) 

(70) 

(71) 

(72) 
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TABLE  n.  SYSTEM  APPROXIMATE  LOWER  CONFIDENCE  BOUNDS  (LCD'S) 
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4. 


RELIABILITY  GROWTH  PROJECTION 


4.1  Reliability  Projection  Concepts  and  Methodology.  The  reliability  growth 
process  applied  to  a  complex  system  undergoing  development  involves  surfacing  failure 
modes,  analyzing  the  modes,  and  implementing  corrective  actions  (termed  fixes)  to  the 
surfaced  modes.  In  such  a  manner,  the  system  configuration  is  matured  with  respect  to 
reliability.  The  rate  of  improvement  in  reliability  is  determined  by  (1)  the  on-going  rate 
at  which  new  problem  modes  are  being  surfaced,  (2)  the  effectiveness  and  timeliness  of 
the  fixes,  and  (3)  the  set  of  failure  modes  that  are  addressed  by  fixes. 

At  the  end  of  a  test  phase,  program  management  usually  desires  an  assessment  of 
the  system’s  reliability  associated  with  the  current  configuration.  Often,  the  amount  of 
data  generated  from  testing  the  current  system  configuration  is  severely  limited.  In  such 
circumstances,  if  the  failure  data  generated  over  a  number  of  system  configurations  is 
consistent  with  a  reliability  growth  model,  we  can  pool  the  data  over  the  tested 
configurations  to  estimate  the  parameters  of  the  growth  model.  This  in  turn  will  yield  a 
reliability  tracking  curve  that  gives  estimates  of  the  configuration  reliabilities.  The 
resulting  assessment  of  the  system’s  current  reliability  is  called  a  demonstrated  estimate 
since  it  is  based  solely  on  test  data. 

If  the  current  configuration  is  the  result  of  applying  a  group  of  fixes  to  the 
previous  configuration,  there  could  be  a  statistical  lack  of  fit  in  tracking  reliability  growth 
between  the  previous  and  current  configurations.  In  such  a  situation  it  may  not  be  valid 
to  use  a  reliability  growth  tracking  model  to  pool  configuration  data  to  assess  the 
reliability  of  the  current  configuration.  We  always  have  the  option  of  estimating  the 
current  configuration  reliability  based  only  on  failure  data  generated  for  this 
configuration.  However,  such  an  estimate  may  be  poor  if  little  test  time  has  been 
accumulated  since  the  group  of  fixes  was  implemented.  In  this  situation,  program 
management  may  wish  to  use  a  reliability  projection  method.  Such  methods  are  typically 
based  on  assessments  of  the  effectiveness  of  corrective  actions  and  failure  data  generated 
from  the  current  and  previous  configurations. 

A  second  situation  in  which  a  reliability  projection  is  often  utilized  is  when  a 
group  of  fixes  are  scheduled  for  implementation  at  the  end  of  the  current  test  phase,  prior 
to  commencing  a  follow-on  test  phase.  Program  management  often  desires  a  projection 
of  the  reliability  that  will  be  achieved  by  implementing  the  delayed  fixes.  This  type  of 
projection  can  be  based  solely  on  the  current  test  phase  failure  data  and  engineering 
assessments  of  the  effectiveness  of  the  planned  fixes.  The  Crow/AMSAA  model  in 
Section  4.3  or  the  AMSAA  Maturity  Projection  Model  (AMPM)  discussed  in  Section  4.4 
can  be  used  to  obtain  such  projections. 

The  current  test  phase  could  consist  of  several  system  configurations  if  not  all  the 
fixes  to  surfaced  problem  modes  are  delayed.  In  this  instance  we  can  still  obtain  a 
projection  of  the  reliability  with  which  the  system  will  enter  the  follow-on  test  by  using 
the  AMPM. 
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Another  situation  in  which  a  projection  can  be  useful  is  in  assessing  the 
plausibility  of  meeting  future  reliability  milestones,  i.e.,  milestones  beyond  the 
commencement  of  the  follow-on  test.  The  AMPM  can  provide  such  projections  b^ed  on 
failure  data  generated  to  date  and  fix  effectiveness  assessments  for  all  implemented  and 
plarmed  fixes  to  surfaced  problem  modes. 

In  Section  4.2  we  present  several  basic  concepts  used  in  connection  with  our 
reliability  projection  models.  We  also  establish  notation  and  present  assumptions  that  are 
used  throughout  this  section.  Notation  and  assumptions  directed  toward  a  particular 
method  are  introduced  in  the  corresponding  section. 

In  Sections  4.3  and  4.4  we  present  two  reliability  projection  models  and 
associated  statistical  procedines.  In  Section  4.3  we  discuss  the  Crow/AMSAA  model 
This  model  is  used  to  estimate  the  system  failure  intensity  at  the  hegiimmg  of  a  follow-on 
test  phase  based  on  infotmation  ftom  the  previous  test  phase.  This  mfoirnation  consists 
of  problem  mode  first  occurrence  times,  the  number  of  failures  associated  with  eac 
problem  mode,  and  the  total  number  of  failures  due  to  modes  that  will  not  be  addressed 
by  fixes.  Additionally,  the  projection  uses  engineering  assessments  of  the  planned 
corrective  actions  to  problem  modes  surfaced  during  the  test  phase.  The  associated 
statistical  estimation  procedure  assumes  that  all  the  corrective  actions  are  implemented  at 
the  end  of  the  current  test  phase  but  prior  to  commencing  the  follow-on  test  phase.  This 
model  addresses  the  continuous  case,  i.e.,  where  test  duration  is  measured  m  a  continuous 

fashion  such  as  in  hours  or  miles. 

In  Section  4.4  we  present  another  reliability  projection  model  that  addresses  the 
continuous  case.  This  model  is  called  the  AMSAA  Maturity  Projection  Model  - 
Continuous  (AMPM-Continuous).  The  model  can  be  applied  to  the  situation  where  one 
wishes  to  utilize  test  data  generated  over  one  or  more  test  phases  to  project  the  impac  o 
fixes  to  surfaced  problem  failure  modes.  The  model  does  not  require  that  the  fixes  be  all 
delayed  to  the  end  of  the  current  test  phase.  It  only  assumes  the  fixes  are  implemented 
prior  to  the  time  at  which  a  projection  is  desired.  Also,  projections  may  be  made  for 
milestones  beyond  the  start  of  the  next  test  phase.  The  section  contains  an  example 
application  of  the  AMPM. 
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4.2  Basic  Concepts,  Notation  and  Assumptions.  Throughout  this  section  we  shall 
regard  a  potential  failure  mode  as  consisting  of  one  or  more  potential  failure  sites  with 
associated  failure  mechanisms.  Fixes  are  often  applied  to  failure  modes  surfaced  through 
testing.  As  in  Reference  [1],  we  shall  define  a  B-mode  to  be  a  failure  mode  we  would 
apply  a  fix  to  if  the  mode  were  surfaced.  All  other  failure  modes  will  be  referred  to  as  A- 
modes.  A  surfaced  mode  might  be  regarded  as  an  A-mode  if  (1)  a  fix  is  not  economically 
justifiable,  or  (2)  the  underlying  failure  mechanisms  associated  with  the  mode  are  not 
sufficiently  understood  to  attempt  a  fix.  Thus  the  rate  of  failure  due  to  the  set  of  A- 
modes  is  constant  as  long  as  the  failure  modes  are  not  reclassified. 

For  a  surfaced  B-mode,  the  rate  of  occurrence  would  hopefully  diminish  after 
implementing  a  fix  to  the  mode.  However,  in  general,  we  cannot  expect  the  mode  rate  of 
occurrence  to  drop  to  zero.  Fixes  are  seldom  perfect;  for  example,  our  fix  may  not 
eliminate  all  the  potential  failure  mechanisms  associated  with  the  B-mode.  Thus,  for 
each  B-mode,  say  mode  i,  we  associate  a  fix  effectiveness  factor  (FEF),  denoted  by  . 

The  FEF  ,  is  the  fraction  by  which  the  initial  rate  of  occurrence  of  mode  i  is  reduced 
due  to  the  fix.  The  assessed  values  for  the  of  surfaced  B-modes  are  often  based 
largely  on  engineering  judgement.  This  is  why  the  corresponding  reliability  assessment 
is  termed  a  “projection”  as  opposed  to  a  “demonstrated  value”  that  is  based  solely  on  the 
test  data. 

List  of  Notation: 

K  Number  of  potential  B-modes  that  reside  in  the  system 
/i,  Initial  rate  of  occurrence  of  B-mode  i  (i  =  1,  •  •  • , 

Contribution  of  A-modes  to  system  failure  intensity 
Ag  B-mode  contribution  to  initial  system  failure  intensity 
T  Total  duration  of  conducted  test.  Typically  measured  in  hours  or  miles. 

N .  Number  of  A-mode  failures  that  occur  over  [0,T] 

Ng  Number  of  B-mode  failures  that  occur  over  [0,T] 

m  Number  of  distinct  B-modes  surfaced  over  [0,T] 

M{t)  Random  variable  of  number  of  distinct  B-modes  surfaced  by  test  duration 

t 

ju{t)  The  expected  value  of  M{t) 

tj  Time  of  first  occurrence  of  B-mode  i  (z  =  1,  •  •  • ,  ) 

t  Vector  of  B-mode  first  occurrence  times  (f, ,  •  •  • ,  ) 

Ni  Number  of  failures  associated  with  B-mode  i  that  occurs  during  test 

d ^  Fix  effectiveness  factor  (FEF)  for  B-mode  i.  The  factor  d-  is  the  fraction 

of  A-  removed  by  the  fix. 

obs  The  index  set  associated  with  the  m  B-modes  that  are  surfaced  during  test 
E  Expectation  operator 

V  Variance  operator 
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mle  Maximum  likelihood  estimator 

When  placed  over  a  parameter,  it  denotes  an  estimate 

~  “Distributed  as” 

«  “Approximated  by” 

=  “Approximately  equal  to” 

Assumptions: 

1 .  At  the  start  of  test,  there  is  a  large  unknown  constant  number,  denoted  by  K, 
of  potential  B-modes  that  reside  in  the  system  (which  could  be  a  complex 
subsystem). 

2.  Failure  modes  (both  types  A  and  B)  occur  independently. 

3.  Each  occurrence  of  a  failure  mode  results  in  a  system  failure. 

4.  No  new  modes  are  introduced  by  attempted  fixes. 

Additional  notation  and  assumptions  germane  to  a  particular  model  will  be 
introduced  in  the  section  dealing  with  the  model. 

4.3  Crow/AMSAA  Reliability  Projection  Model. 

4.3.1  Introduction.  In  this  section  we  shall  consider  the  case  where  all  fixes  to 
surfaced  B-modes  are  implemented  at  the  end  of  the  current  test  phase  prior  to 
commencing  a  follow-on  test  phase.  Thus  all  fixes  are  delayed  fixes.  The  current  test 
phase  will  be  referred  to  as  Phase  I  and  the  follow-on  test  phase  as  Phase  II. 

The  Crow/AMSAA  reliability  projection  model  and  associated  parameter 
estimation  procedure  was  developed  to  assess  the  reliability  impact  of  a  group  of  delayed 
fixes.  In  particular,  the  model  and  estimation  procedure  allow  assessment  of  what  the 
system  failure  intensity  will  be  at  the  start  of  Phase  II  after  implementation  of  the  delayed 
fixes.  Denoting  this  failure  intensity  by  r(T),  where  T  denotes  the  duration  of  Test  Phase 
I,  the  Crow/AMSAA  assessment  of  r(T)  is  based  on:  (1)  the  A  and  B  mode  failure  data 
generated  during  Phase  I  test  duration  T;  and  (2)  assessments  of  the  fix  effectiveness 
factors  (FEFs)  for  the  B-modes  surfaced  during  Phase  I.  Since  the  assessments  of  the 
FEFs  are  often  largely  based  on  engineering  judgement,  the  resulting  assessment,  f(r), 
of  the  system  failure  intensity  after  fix  implementations  is  called  a  reliability  projection 
as  opposed  to  a  demonstrated  assessment  (which  would  be  based  solely  on  test  data). 

The  Crow/AMSAA  projection  model  and  estimation  procedure  was  motivated  by 
the  desire  to  replace  the  widely  used  “adjustment  procedure.”  The  adjustment  procedure 
assesses  r(T)  based  on  reducing  the  number  of  failures  due  to  B-mode  i  during  Phase 

I  to  1  —  d’  Nj ,  where  d*  is  the  assessment  of  d^ .  Note  (l  —  d^  )a/,.  is  an  assessment  of 

V  J 
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the  expected  number  of  failures  due  to  B-mode  i  that  would  occur  in  a  follow-on  test  of 
the  same  duration  as  Phase  I.  The  adjustment  procedure  assesses  r(T)  by  where 


Crow[l]  shows  that  even  if  the  assessed  FEFs  are  equal  to  the  actual  cf, ,  the 
adjustment  procedure  systematically  underestimates  r(T).  This  bias,  i.e., 


B{t)  =  E{r{T)-K,j{T)\  >  0 


(2) 


is  calculated  in  [1]  by  considering  the  random  set  of  B-modes  surfaced  dunng  Phase  I.  In 
particular,  the  adjustment  procedure  is  shown  to  be  biased  since  it  fails  to  t^e  into 
account  that,  in  general,  not  all  the  B-modes  will  be  surfaced  by  the  end  of  Phase  I 
Before  discussing  how  the  Crow/AMSAA  methodology  addresses  this  bias  we  shall  list 
some  additional  notation  and  assumptions  associated  with  the  Crow/AMSAA  model. 

4.3.2  Crow/AMSAA  Model  Notation  and  Additional  Assumptions. 


List  of  Notation: 


Di 


At) 

fjT) 

B(T) 


Pop 

M  Qp 


The  conditional  random  variable  for  B-mode  i  (i  -  1, . . .,  K)  whose 
realization  is  the  fix  effectiveness  factor  di  if  mode  i  occurs  during  Test 

Phase  I. 

Expected  value  of  Di 
Length  of  Test  Phase  I 

System  failure  intensity  at  beginning  of  Test  Phase  II  after  implementation 
of  delayed  B-mode  fixes.  Viewed  as  a  random  variable  whose  value  is 
determined  by  the  set  of  B-modes  surfaced  during  Test  Phase  I  and  the 
associated  fix  effectiveness  factors. 

Expected  value  of  r(T)  with  respect  to  random  set  of  B-modes  surfaced  m 
Test  Phase  I,  conditioned  on  the  fix  effectiveness  factor  values.  We  write 
p{T)=E[r{T)). 

Adjustment  procedure  assessment  of  the  value  taken  on  by  r(T) 

Bias  incurred  by  assessing  the  value  of  r(T)  by  r^j  (T).  Thus, 

s(r)=£Hr)-r.«(J')) 

Growth  potential  system  failure  intensity 
Growth  potential  system  MTBF,  i.e.,  M^jp  =  {pcp) 
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h{f)  Expected  rate  of  occurrence  of  new  B-modes  at  test  duration  t.  Note 
a  t 

h  (O’  ''cCO’  Pc^)  Crow/AMSAA  model  approximations  to  h{t),  r(t),  pif) 
respectively 

M(T),  Mc(T)  Denote  (/^(T’))  '  and  {pdT))  respectively 


Additional  Assumptions  for  Crow/AMSAA: 

1 .  The  time  to  first  occurrence  is  exponentially  distributed  for  each  failure  mode. 

2.  No  fixes  to  B-modes  are  implemented  during  Test  Phase  I.  Fixes  to  all  B- 
modes  surfaced  during  Phase  I  are  implemented  prior  to  Phase  II. 

3.  The  fix  effectiveness  factors  (FEFs)  d,  associated  with  the  B-modes  surfaced 
during  Phase  I  are  realized  values  of  the  random  variables  Di  (i  -  1,  • . K) 
where: 

(a)  The  D,  are  independent; 

(b)  The  Z),  have  common  mean  value  ;  and 

(c)  The  D-  are  independent  of  M(r) . 


4  The  random  process  for  the  number  of  distinct  B-modes  that  occur  over  test 
interval  [o,t],  i.e.  M{t),  is  well  approximated  by  a  non-homogeneous  Poisson 

process  with  mean  value  function  =  for  some  A,^>0. 


4.3.3  Crow/AMSAA  Model  Equations  and  Estimation  Procedure.  The 

Crow/AMSAA  model  assesses  the  value  of  the  system  failure  intensity,  r(T),  after 
implementation  of  the  Phase  I  delayed  fixes.  This  assessment  is  taken  to  be  an  estimate 
of  the  expected  value  of  r(T),  i.e.,  an  estimate  of  p{T)  =  E{r{T)).  In  [1]  (and  m  Section 

4.4.3)  it  is  shown  that: 


At)  = 

1=1 


-/i,  r 


(3) 


The  traditional  adjustment  procedure  assessment  for  the  value  of  r(T)  is  actually  an 
estimate  of 


i=i 

since  as  shown  later  in  this  subsection 
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(4) 


i=\ 

where  df  is  an  assessment  of  rf, .  Thus,  by  (3)  and  (4),  the  adjustment  procedure  has  the 
bias  B(T)  where 

B(T)  = 

=  p{T)-E(f,^{T)) 

/=!  »=1 

It  follows  that  for  cf,'  =  tf,  (i  =  1,  ■  ■  ■ , 

5(r)  =  'Zd,A,e-‘'^  (5) 

/=! 

This  shows  that  even  with  perfect  knowledge  of  the  di  (i.e.,  when  d-  =  d^ ),  the 

adjustment  procedure  provides  a  biased  underestimate  of  the  value  of  r(T).  The 
Crow/AMSAA  procedure  attempts  to  reduce  this  bias  by  estimating  B(T)  given  by  (5). 

To  estimate  B(T),  the  Crow/AMSAA  Model  uses  an  approximation  to  B(T).  This 
approximation  is  obtained  in  two  steps.  The  first  step  is  to  regard  the  d^  in  (5)  as 

realizations  of  random  variables  Z),  (/  =  1,  •  •  • ,  Zf)  that  satisfy  assumption  number  3  in  the 
“Additional  Assumptions  for  Crow/AMSAA.”  Then  B(T)  is  approximated  by  the 
expected  value  (with  respect  to  the  D.)  of 

/=! 

Thus  the  initial  approximation  arrived  at  for  B(T)  in  (5)  is 

B{t)  * 

V  /=!  / 

=  («) 
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where  /Ij=E{Di)  {i  =  \,---,K).  The  final  step  to  obtain  the  Crow/AMSAA 
approximation  of  B(T)  is  to  replace  the  sum 

i=\ 

in  (6)  by  a  two  parameter  fimction  of  T.  The  Crow/AMSAA  Model  replaces  this  sum  by 
the  power  function 


K{t)  =  Aj3T^-'  for  A,J3>0  (7) 

The  form  in  (7)  is  chosen  based  on  the  desire  for  a  mathematically  tractable  estimation 
problem  and  an  empirical  observation.  Based  on  an  empirical  study,  Crow  [1]  states  that 
the  number  of  distinct  B-modes  surfaced  over  a  test  period  [O,/]  can  often  be 
approximated  by  a  power  function  of  the  form 

^^{t)  =  for  A,fi>0  (8) 

In  (8),  Crow  [1]  interprets  ju^{t)  as  the  expected  number  of  distinct  B-modes  surfaced 
during  the  test  interval  [o,t].  More  specifically,  [1]  assumes  the  number  of  distinct  B- 
modes  occurring  over  [o,  t]  is  governed  by  a  non-homogeneous  Poisson  process  with 
//^  (/)  as  the  mean  value  function.  Thus 

*  (,)  =  (9) 

dt 

represents  the  expected  rate  at  which  new  B-modes  are  occurring  at  test  time  t. 

In  Annex  1  of  Appendix  D,  under  the  previously  stated  assumptions,  it  is  shown 
that  the  expected  number  of  distinct  B-modes  surfaced  over  [0,t]  is  given  by 

Ml)  =  (•») 

/=! 

Thus  the  expected  rate  of  occurrence  of  new  B-modes  at  test  time  t  is 

=  Ml  =  (11) 

d  t  M 

Equation  (11)  shows  that  the  initial  approximation  to  the  bias  B(T),  given  in  (6)  can  be 
expressed  as 
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(12) 


B(T)  » 

By  replacing  h(T)  in  (12)  by  hfi)  given  in  (9),  we  arrive  at  the  final  Crow/AMSAA 
Model  approximation  to  B(T),  namely 

5,(r)  = 

(13) 


X 


Returning  to  our  expression  in  (3)  for  the  expected 
intenstty  after  inUcration  of  the  Phase  I  delayed  fixes,  t.e  4^)=  MKj)). 
now  write  down  the  Crow/AMSAA  Model  approximation  for  p(T) .  This 

approximation,  by  (13),  is  given  by. 

pXt)  =  + 


(14) 


/=1 


We  shall  next  consider  the  Crow/AMSAA  procedure  for  estimating  p,(T)  -  Th's 

estimate  is  taken  as  the  assessment  of  the  system  failure  intensity  after  incorporation 
the  delayed  fixes. 

Consider  the  first  term  in  the  expression  for  pM  S"®"  “  ('''>•  ’ 

the  A-modes  are  not  fixed,  the  A-mode  failure  rate  is  constant  over  I0,T].  Thus  we 

simply  estimate  by 


=  Y 


(15) 


where 


N  is  the  number  of  A-mode  failures  over  [0,T].  Note 


E{N,)  ^aT  _ 


(16) 


Next  consider  estimation  of  the  summation 

±(\-d,)X, 
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in  the  expression  for  (r) .  By  the  second  assumption  in  the  “Additional  Assumptions 
for  Crow/AMS  AA,”  all  fixes  are  delayed  until  Test  Phase  I  has  been  completed.  This 
implies  the  failure  rate  for  B-mode  i  (i  =  1,  •  •  • ,  /:)  remains  constant  over  [0,T].  Thus  we 

simply  estimate  /?,•  by 


0=1, 


(17) 


where  A,-  denotes  the  number  of  failures  during  [0,T]  attributable  to  B-mode  i.  Note 


E 


E{N.)  ^  ^ 
T  T 


(18) 


Equations  (16)  and  (18)  suggest  we  assess 

/=! 


by 


r^,(T)  = 


Aa  + 


/=1 


Observe  A,  =  0  if  B-mode  i  does  not  occur  during  [0,T].  Thus 


(19) 


E(i 

ieobs 


(20) 


where  obs  =  {i  |  B-mode  i  occurs  during  [0,T]}.  Note  the  adjustment  procedure  estimate 
has  the  form 


where 

ieobs 


(21) 


(22) 
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is  the  “adjusted”  number  of  failures. 

For  given  fix  effectiveness  factor  (FEF)  assessments,  < ,  note  that 

=  r‘kr  +  i(i-<i;)(i,r)| 

=  +  (23) 

1=1 

Thus,  as  stated  earlier,  we  see  that  the  adjustment  procedure  estimate  only  provides  an 
assessment  for  a  portion  of  the  expected  system  failure  intensity,  namely 

(=1 

Returning  to  the  fundamental  equation  for  the  Crow/AMSAA  Model 
approximation  to  the  expected  system  failure  intensity,  i.e.  (14), 

pAt)  =  4  + 

/=1 

Let  us  next  consider  the  assessment  of  the  fix  effectiveness  factors  d, .  The  assessment 
df  will  often  be  based  largely  on  engineering  judgement.  The  value  chosen  for  <  should 

reflect  several  considerations:  ,  •  i  .  r-  j. /ox 

(1)  How  certain  we  are  that  the  root  cause  for  B-mode  i  has  been  correctly  identified,  (  ) 
the  nature  of  the  fix,  e.g.,  its  complexity;  (3)  past  FEF  experience;  and  (4)  any  germane 
testing  (including  assembly  level  testing). 

Note  that  (20)  shows  that  we  need  only  assess  FEFs  for  those  B-modes  that  occur 
during  [0,T]  to  make  an  assessment  of 

/=! 

To  assess  the  mean  FEF,  ju,  =  E{D,  ) ,  we  utilize  our  assessments  d.  for  i  e  obs . 
Let  m  be  the  number  of  distinct  B-modes  surfaced  over  [0,T].  Then  we  assess  by 
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(24) 


;  = 


^  ie.obs 


To  complete  our  assessment  of  the  expected  system  failure  intensity  after 
incorporation  of  delayed  fixes,  we  shall  now  address  the  assessment  of 

hXT)  = 

To  develop  a  statistical  estimation  procedure  for  X.  and  (3,  the  \ 

regards  the  number  of  distinct  B-modes  occurring  in  an  interval  [0,t],  denoted  by  M(  ) , 

as  a  random  process.  The  model  assumes  that  this  random  process  can  be  well 
approximated,  for  large  K,  by  a  non-homogeneous  Poisson  process  with  mean  va 

function 

^^(,)  =  E(M{f))  = 


where  X,  p,  t  >  0.  As  noted  earlier  in  (9) 

Kit)  = 


dMcif) 


dt 


The  data  required  to  estimate  >.  and  p  are  (1)  the  number  of  distinct  B-modes,  m, 

that  occur  during  rO,T]  and  (2)  the  B-mode  first  occurrence  times 

0  <  <  ^2  ^  ^  ^  maximum  likelihood  estimates  of  X  an 

P,  denoted  by  ^  and  >  respectively,  satisfy  the  following  equations: 

d  (25) 

Xt^  =  m 


P  = 


m 


m  Y 


(26) 


Note  1253  merely  says  that  the  estimated  number  of  distinct  B-modes  that  occm  dunng 
[0  T]  should  equal  the  observed  number  of  distinct  B-modes  over  this  penod.  Solving 
(25)  for  i  we  can  write  our  estimate  for  /i,(r)  in  terms  of  m  and  as  follows. 


hXT)  = 


-1  — 


m 


PTP- 


m  P 

~T 


(27) 
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Crow  [1]  notes  that  conditioned  on  the  observed  number  of  distinct  B-modes  m , 
i.e.  U{T)  =  w ,  the  estimator 


f— 1 

^  m  j 


A 

■P 


m>2 


(28) 


is  an  unbiased  estimator  of  p,  i.e., 

fife)  =  p  m 

Thus  we  shall  also  consider  estimating  =  XpT^~^  by  using  .  This  leads  to 

the  estimate 


*,(?•)  = 


mP. 

T 


(30) 


Finally,  to  complete  our  assessment  of  the  system  failure  intensity,  we  need  to 
assess  the  Crow/AMSAA  Model  expected  system  failure  intensity  Recall,  by 


(14) 


P,(T)  = 

i=\ 


(31) 


Piecing  together  our  assessments  for  the  individual  terms  in  (3 1)  we  arrive  at  the 
following  assessment  for  p^i^)  based  on  p : 


Since  =  0  for  i  i  obs ,  we  finally  obtain 


pAt)  = 

^  t  ieobs  ieobs  J 


(32) 


Likewise,  we  arrive  at  the  following  alternate  assessment  for  {T )  based  on  P„ 

(provided  m>2): 
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(33) 


A(T)  =  7jw.  +  S(i-<K+AE<'} 

^  L  i^obs  leobs  j 

Note  both  estimates  of  (T )  are  of  the  form 

Estimate p^{t)  =  —  j  A^*  +  {estimate I  (34) 

T  (  i&obs  J 

where  N*  is  the  “adjusted”  number  of  failures  over  [0,T].  Recall  the  historically  used 
adjustment  procedure  assessment  for  the  system  failure  intensity,  after  incorporation  of 
delayed  fixes,  is  given  by 


radjV)  =  — 

Also  recall 


Thus  we  see  by  (32)  and  (33) 

<  A(r)  <  pAt)  (35) 

Also  of  interest  is  an  assessment  of  the  reciprocal  of  i-e- 


m.(7-)  =  \pXt)Y 

The  assessment  for  the  system  mean  time  between  failures  after  incorporation  of  the 
delayed  fixes,  denoted  by  M(T),  is  taken  to  be  the  Crow/AMSAA  Model  assessment  of 
M^(r).  The  assessments  of  Mcfl)  based  on  p^^)  and  are  denoted  by  M^(7’) 

and  Mc(7’)  respectively.  Thus 

K{T)  =  {pAt)Y'  (36) 

and 

M.M  =  fe(J')}"  (33) 

By  (35)  we  have 
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(38) 


M^(r)  <  M,(r)  < 

In  Section  4.3.5  we  shall  argue  that  ^,(r)  generally  provides  a  more  accurate 
assessment  of  pXt)  than  does  A(^)-  However,  somewhat  surprisingly  at  first  thought, 
in  Section  4.3.5  we  identify  conditions  under  which  M,(r)  generally  provides  a  more 
accurate  assessment  of  M^(7’)  than  does  M^(7’). 

4.3.4  Reliability  Growth  Potential.  Consider  the  expression  in  (3)  for  p{T),  the 

expected  system  failure  intensity  after  incorporation  of  the  delayed  fixes.  If  we  let 
r  ->  00  and  denote  the  resulting  limit  of  p{T)  by  Pap  we  obtain 

p  =  \im  p{T)  = 

r-»®  ,=i 


The  expression  Pop  is  called  the  growth  potential  failure  intensity.  Its  reciprocal  is 
referred  to  as  the  growth  potential  MTBF.  The  growth  potential  ^ 

factors.  Note  Pop  is  estimated  by 


Pgp 


(40) 


ieobs 


If  the  reciprocal  W)"  lies  below  the  goal  MTBF  thenthis  may  indicate  that  achieving 
the  goal  is  high  risk. 

4.3.5  Use  of  the  Maximum  Likelihood  Estimator  versus  the  Unbiased  Estimator 

for  p.  Recall  that  the  estimator 


conditioned  on  M(r)=  w ,  with  m  >  2 ,  is  unbiased  for  p,  i.e. 

fife)  =  P 

Furthermore  the  variances  of  A  and  ^.denoted  by  rfe)  and  Kfe  respectively, 
satisfy  the  following: 
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v{A,)  =  >' 


=  vis)  <  ri^)  (41) 

\  m  J 

for  w  >  2 .  Equation  (41)  together  with  the  unbiased  property  of  ,  suggest  that 
provides  a  more  accurate  assessment  of  P  than  does  P . 

Next  consider  the  assessments  of  (r )  based  on  P  and  p^ .  Recall  the 
Crow/AMSAA  Model  assumes  that  M(t),  t>0,  is  a  non-homogeneous  Poisson  process 
with  mean  value  function 

=  E(M{t))  =  ^,P>^ 

Thus,  in  particular,  M(T)  is  Poisson  distributed  with  mean 

E{m{t))  = 

Using  this  fact,  it  can  be  shown  that  A^(r)  is  an  approximately  unbiased  estimator  of 
hXT)  under  most  conditions  of  practical  interest,  where  it  is  understood  that  h^T) 
denotes  a  conditional  estimator,  conditioned  on  M{t)>2  .  To  be  more  explicit,  h^{T) , 
when  viewed  as  an  estimator  (as  opposed  to  an  estimated  value),  is  a  random  variable 
which  is  a  function  of  M(T)  and  the  random  vector  of  B-mode  first  occurrence  times 

When  M{T)  =  m  and  =  the  estimator  h^T)  takes 

m  P„ 

on  the  value - 

T 

where 


The  estimator  h^  ip)  can  be  shown  to  satisfy  the  following: 

£(*,(/))  =  *,(r)  (42) 
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provided  Pr(M(7’)  =  O)  =  0 ,  where  Pr  denotes  the  probability  function  for  M(T). 


Consider  the  variances  of  the  estimators  (r )  and  (r )  conditioned  on 

M{r)  =  m.  Vox  m>2. 


v[hXT)\M{T)=m)  =  V 


V  T  J 


.  (?) 


(( 


'"-'V 


m 


T 


v-*  y 


'{fi)  = 


m-l 


V 


iA) 


kTj 

v[hXT)\M(T)  =  m] 


v{^)  =  V 


kT  j 


(43) 


Now  consider  the  variances  of  and  h^{r)  conditioned  on  M{t)>2  .  Since  (43) 
holds  for  each  m  >  2 ,  we  have 


v(k{t)\m{t)>2)  <  v(hXT)\M{T)>2) 


(44) 


Equations  (42)  and  (44)  suggest  that  the  estimator  h^{T)  provides  a  more  accurate 

estimate  of  ^<.(7’)  than  does  the  estimator  h^T)  when  two  or  more  distinct  B-modes 
occur  during  [0,T]. 

We  now  investigate  the  bias  of  the  estimators  {T  )  and  p^iT).  To  do  so,  let 

pAt)  =  + 


ieobs 


ieobs 


where  Also  let  (t)  =  .  By  (27)  and  (30)  we  have 
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Mt)  = 


ieobs 


mP 

J 


Thus  the  expected  value  of  pX^')  is 


Ei?.iT))  =  i.+id-rf.-M+fiU^W)  (‘*5) 


/=! 


Recall  by  (31), 


PciT)  =  ^A+tli^-di)^i+P,hXT) 


i=\ 


Thus  by  Equation  (45),  we  have 

e(?At))-pAt)  = 

-d')^,  +  -Ad)i>.  (T)) +aJ£(*.  (?■))- A,  (o! 


/=1 


(46) 


By  (46)  we  see  that  even  if  our  assessments  of  pd  and  the  dj  are  perfect,  the  estimator 
PciT)  will  have  a  residual  bias  of 

P,  {E(*;(7-))-/.,(r)} 

To  reduce  this  residual  bias  as  much  as  possible,  we  wish  to  make  the  bias 

as  small  as  possible.  Since  (T )  is  almost  an  unbiased  estimator  for  (t)  ,  this 

suggests  we  use 

pXt)  =  +  Z (i " 

1  [  ieobs  i^obs 


to  assess  p^ij'Y 
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Next ,  we  discuss  the  assessment  of  M^{r)  =  ‘  •  To  do  so  let 

KiT)  =  ■  Thus 


M,(r) 


i  t  i€0b5  ieobs 


-I 


^  i  K^ieobs  J 


n-1 


ieobs 


(47) 


Also 


M,(r)  = 


J  =  l 


-1 


(48) 


We  have  shown  that  to  minimize  the  bias 

Efe(T))-/t,(T) 


we 


should  use  pX'T)  instead  of  pX^)  to  estimate  p^T).  However,  we  wish  to 
demonstrate  that  one  should  not  infer  from  this  that  M^(7')  must  have  a  smaller  bias  than 
M^(r)  as  an  estimator  of  M,(r).  To  demonstrate  this  we  shall  consider  a  simple  case, 

the  instance  when  the  bias  of  M^(r)  is  approximately  equal  to  the  bias  of  ^(t)}  . 


Thus  in  the  following  assume 


e[m,(T)1M(T)>2]  -  M,(T) 

-  K(T)}-‘ 


(49) 


One  instance  where  (49)  would  be  expected  to  hold  is  when  Xa  =  0  and  dj  s  1  for  i  -  1 , 

. . .,  K.  For  such  conditions,  we  have  by  (48)  that  Mc(T)  s  {he  (T)}  *.  Also,  in  such  a 

case,  it  is  reasonable  that  dj  =  1  for  i  e  obs  and,  with  high  probability,  ^  =  0  •  By  (47) 
we  see  that  such  conditions  would  imply 
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e[m,(T)1A/(T)>2] 

=  E  5i,(T))"'|Af(T)>2  . 

The  above  expectations  and  all  subsequent  expectations  in  this  section  are  with  respect  to 
all  the  random  quantities  for  given  d’ ,  conditioned  on  M(T)  >  2.  These  random 
quantities  are  the  number  of  A-mode  failures  and  the  number  of  distinct  B-modes 
experienced  over  [0,  T],  and  the  random  vector  of  B-mode  first  occurrence  times 
(Ti, Ta/(t))- 

Now  consider  the  expected  values  of  ^  (T)|^  and  {h^  (T)}  conditioned  on 
M(T)  >  2.  From  the  fact  that  the  number  of  distinct  B-modes  occurring  over  [0,T]  is 
Poisson  with  mean  A,T^  it  can  be  shown 

{h,(r)r' <  E[{*,(7-)r'|M(T)2  2] 

<  E[ft(r)r'|M(T)k2j  (50) 

for  p(T)  >  3.2.  Thus,  when  (49)  holds,  Equation  (50)  implies 

e[m,(T)|M(T)>2J-M,  (T) 

s  E  [^,(T)}'|M(T)>2]  -{h,(r)r' 

>  E  [Ji,(T)}‘|M(T)>2j  -{h,(r)}-' 

-  E  [m,  (T)|M(T)>2j  -M,(r)  (51) 

and 

e[m,  (T)|M(T)>2j-M,  (T) 

=  E  (T)}’’  |M(T)>2  -{h,(r)}-'  >  0  (52) 
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Equations  (51)  and  (52)  show  that  for  the  case  considered  we  should  anticipate 
that  M^(7’)  and  Mc(r)  will  have  positive  biases  with  the  bias  of  M^{t)  larger  than  that 
of  M^T).  It  has  not  been  established  whether  this  holds  more  generally  .  If  there  is 
concern  that  M^r)  will  have  a  positive  bias  and  that  the  bias  of  M^(r)  will  exceed  that 
of  M,(r),  then  one  may  wish  to  assess  M^(r)  by  the  more  conservative  estimator 
M,(r)  (recall  M,(r)  <  M,(r)  for  M(T)  >2). 

4.3.6  Example.  The  following  example  is  taken  from  [1]  and  illustrates  application  of 
the  Crow/ AMS AA  model. 

Data  were  generated  by  a  computer  simulation  with  =  0.02 ,  Ag  =  0.1 , 
a:  =  100  and  the  </,.  ’s  distributed  according  to  a  beta  distribution  with  mean  0.7.  The 
simulation  portrayed  a  system  tested  for  T  =  400  hours.  The  simulation  generated 
N  =  42  failures  with  =  10  and  Ng  =  22.  The  thirty-two  B-mode  failures  were  due 
to  M=16  distinct  B-modes.  The  B-modes  are  labeled  by  the  index  i  where  the  first 
occurrence  time  for  mode  i  is  t.  and  0<t,  <^2  <T  =  400 . 

Table  1  lists,  for  each  B-mode  i,  the  time  of  first  occurrence  followed  by  the  times 
of  subsequent  occurrences  (if  any).  Column  3  of  the  table  lists  A/,- ,  the  total  number  of 

occurrences  of  B-mode  i  during  the  test  period.  Column  4  contains  the  assessed  fix 
effectiveness  factors  for  each  of  the  observed  B-modes.  Column  5  has  the  assessed 
expected  number  of  type  i  B-modes  that  would  occur  in  T=400  hours  after 
implementation  of  the  fix.  Finally,  the  last  column  contains  the  base  e  logarithms  of  the 

B-mode  first  occurrence  times.  These  are  used  to  calculate  . 
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Table  1.  Projection  Example  Data. 


B-mode 

Failure  Times  (hrs) 

d\ 

Inr,- 

1 

15.04,  254.99 

2 

.67 

.66 

2.7107 

2 

25.26, 120.89.  366.27 

3 

.72 

.84 

3.2292 

3 

47.46,  350.2 

2 

.77 

.46 

3.8599 

4 

53.96,315.42 

2 

.77 

.46 

3.9882 

5 

56.42,  72.09.  339.97 

3 

.87 

.39 

4.0328 

6 

99.57,  274.71 

2 

.92 

.16 

4,6009 

7 

100.31 

1 

.50 

.50 

4.6083 

8 

111.99,263.47.373.03 

3 

.85 

.45 

4.7184 

9 

125.48, 164.66.  303.98 

3 

.89  ^ 

J3 

4.8321 

10 

133.43, 177.38,  324.95,  364.63 

4 

.lA 

1.04 

4.8936 

I  \J 

II 

192.66 

1 

.70 

.30 

5.2609 

12 

249.15,324.47 

2 

.63 

.74 

5.5181 

13 

285.01 

1 

.64 

.36 

5.6525 

14 

379.43 

1 

.  .72 

.28 

5.9387 

15 

388.97 

1 

.69 

.31 

5.9635 

16 

395.25  _ 

1 

.46 

.54 

5.9795 

1  V/ 

Totals 

32 

11.54 

7.82 

75.7873 

From  Equation  (1)  and  Table  1,  the  adjustment  procedure  estimate  of  r(T) 


r(400)  is 


=  =  0.04455 

400 

Thus  the  adjustment  procedure  estimate  of  the  system  MTBF  is 

(«»)}■'  =  ^ 

Looking  at  Equation  (40),  we  can  see  that  the  adjustment  procedure  estimate  of  system 
failure  intensity  after  implementation  of  the  fixes  is  simply  ^gp  >  estimate  growt 

potential  failure  intensity.  Thus 

=  *=w/‘'00)  =  0.04455 


Also,  the  estimate  of  the  system  growth  potential  MTBF  is 


^  —l 

Pgp 


k«(«0))-'  =  22.45 
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Stwe ctl/^SAA  model  estimation  e,ua.ion  (32).  This  projectton  ts 
given  by 


^(400)  -  Pgp^ 


0.04455+  ^  (11-54) 
1  400  I 


The  mle  p  is  obtained  from  Equation  (26),  i.e., 


yin  - 

U,- 


anT-^\nti 


16  In  400  -  75.7873 


=  0.7970 


Thus,  by  (53),  the 


Crow/AMSAA  projection  for  the  system  failure  intensity,  based  on  p. 


^^(400)  =  0.04455 


0.7970 


(11.54) 


=  0.06754 

The  corresponding  MTBF  projection  is 

A  nearly  unbiased  assessment  of  the  system  failure  intensity,  for  d'  =  rf,,  can  be 
obtained  by  using  instead  of  p .  Recall  by  (28), 
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By  Equation  (33),  the  projected  system  failure  intensity  based  on  is 


:(400)  = 


^  ieobs 


=  0.04455  + 


0.7472 


V 


400  J 


1(11-54) 


=  0.06611 


The  corresponding  MTBF  projection  is 

fe(400)r'  =  15.13 

As  discussed  in  Section  A3.5,  we  recommend  basing  the  projected  system  failure 
intensity  on  p,  (t)  which  uses  ,  but  assess  the  projected  system  MTBF  by  using  p. 
Thus  in  this  example  we  would  recommend  assessing  the  projected  system  failure 
intensity  by 

^(400)  =  0.06611 


and  the  projected  system  MTBF  by 

{pAm)]-'  =  14.81 


4.4 


The  AMSAA  Maturity  Projection  Model  (AMPM)  -  Continuous. 


4.4.1  Introduction.  The  continuous  version  of  the  AMPM  assumes  the  test 

duration  is  measured  in  a  continuous  scale  such  as  time  or  miles.  Tluoughout  this  section 
AMPM  will  refer  to  the  continuous  version  of  the  model  and  we  shall  refer  to  time  as  t 
measure  of  test  duration. 

The  AMPM  addresses  making  reliability  projections  in  several  situations  of 
interest.  One  case  corresponds  to  that  addressed  by  the  Crow/AMSAA  P^ojectmn  mo  e 
introduced  in  [1]  and  discussed  in  Section  4.3.  This  is  the  situation  in  which  all  fixes  to 
B-modes  are  implemented  at  the  end  of  the  current  test  phase,  Phase  I,  poor  to 
commencing  a  follow-on  test  phase,  Phase  II.  The  projection  problem  is  to  assess  the 
expected  system  failure  intensity  at  the  start  of  Phase  II.  Another  situahon  handled  by 
AMPM  estimation  procedure  is  the  case  where  the  reliability  of  the  unit  mder  test  has 
been  maturing  over  Test  Phase  I  due  to  implemented  fixes  dunng  Phase  I.  This  case 
includes  the  situations  where 
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(i)  all  surfaced  B-modes  in  Test  Phase  I  have  fixes  implemented  within  this  test 
phase  or 

(ii)  some  of  the  surfaced  B-modes  are  addressed  by  fixes  within  Test  Phase  I  and  the 
remainder  are  treated  as  delayed  fixes,  i.e.,  are  fixed  at  the  conclusion  of  Test  Phase  I, 
prior  to  commencing  Test  Phase  II. 

A  third  type  of  projection  of  interest  involves  projecting  the  system  failure 
intensity  at  a  future  program  milestone.  This  future  milestone  may  occur  beyond  the 
commencement  of  the  follow-on  test  phase. 

All  the  above  type  of  projections  are  based  on  the  Phase  I  B-mode  first  occurrence 
times,  whether  the  associated  B-mode  fix  is  implemented  within  the  current  test  phase  or 
delayed  (but  implemented  prior  to  the  projection  time).  In  addition  to  the  B-mode  first 
occurrence  times,  the  projections  are  based  on  an  average  fix  effectiveness  factor  (FEF). 
This  average  is  with  respect  to  all  the  potential  B-modes,  whether  surfaced  or  not. 
However,  as  in  the  Crow/ AMS AA  model,  this  average  FEF  is  assessed  based  on  the 
surfaced  B-modes.  For  the  AMPM  model,  the  set  of  surfaced  B-modes  would  typically 
be  a  mixture  of  B-modes  addressed  with  fixes  during  the  current  test  phase  as  well  as 
those  addressed  beyond  the  current  test  phase. 

In  some  instances,  a  reliability  projection  for  a  future  milestone  can  be  based  on 
extrapolating  a  reliability  growth  tracking  curve.  Such  a  curve  only  utilizes  cumulative 
failure  times  and  does  not  use  B-mode  fix  effectiveness  factors.  This  is  a  valid  projection 
approach  provided  it  is  reasonable  to  expect  that  the  observed  pattern  of  reliability 
growth  will  continue  up  through  the  milestone  of  interest.  However,  this  pattern  could 
change  in  a  pronounced  manner.  Reasons  for  such  a  change  include 

(i)  a  change  in  the  test  environment; 

(ii)  a  different  level  of  future  resources  to  analyze  and  implement  effective 
corrective  actions;  and 

(iii)  jumps  in  reliability  due  to  delayed  fixes. 

If  extrapolating  the  current  tracking  curve  is  not  deemed  suitable  due  to  considerations 
such  as  above,  the  AMPM  projection  methodology  may  be  useful.  Unlike  assessments 
based  on  the  tracking  model,  the  AMPM  assessments  are  independent  of  the  fix 
discipline,  as  long  as  the  fixes  are  implemented  prior  to  the  projection  milestone  date  of 
interest.  Unlike  the  reliability  growth  tracking  model  in  [2],  the  AMPM  (as  well  as  the 
Crow/ AMS  AA  projection  model)  utilizes  a  non-homogeneous  Poisson  process  with 
regard  to  the  number  of  distinct  B-modes  that  occur  by  test  duration  t.  The  associated 
pattern  of  B-mode  first  occurrence  times  is  not  dependent  on  the  corrective  action 
strategy,  under  the  assumption  that  corrective  actions  are  not  inducing  new  B-modes  to 
occur.  Thus  the  AMPM  assessment  procedure  is  not  upset  by  jumps  in  reliability  due  to 
delayed  groups  of  fixes.  In  contrast,  reliability  growth  tracking  curve  methodology 
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utilizes  the  pattern  of  cumulative  failure  times.  Such  a  pattern  is  sensitive  to  the 
corrective  action  strategy.  Thus  a  reliability  growth  tracking  curve  model  may  not  be 
appropriate  for  fitting  failure  data  or  for  extrapolating  due  to  a  corrective  action  strategy 
that  is  not  compatible  with  the  model. 

Note  that  AMPM  reliability  projections  for  a  future  milestone  would  be  optimistic 
if  corrective  actions  beyond  the  current  test  phase  were  less  effective  than  the  average 
FEF  assessment  based  on  B-modes  surfaced  through  the  current  test  phase.  Also,  a 
change  in  the  future  testing  environment  could  result  in  a  new  set  of  potential  failure 
modes  or  affect  the  rates  of  occurrence  of  the  original  set  of  failure  modes.  Either  of 
these  circumstances  would  tend  to  degrade  the  accuracy  of  the  AMPM  reliability 
projection. 

Another  instance  in  which  a  reliability  projection  model  would  be  useful  is  when 
the  current  test  phase  contains  a  number  of  design  configurations  of  the  units  under  test 
due  to  incorporation  of  reliability  fixes  during  the  test  phase.  If  there  is  a  lack  of  fit  of  the 
reliability  growth  tracking  model  over  these  configurations  then  the  tracking  model 
should  not  be  used  to  assess  the  reliability  of  the  latest  configuration  or  for  extrapolation 
to  a  future  milestone.  Such  a  lack  of  fit  may  be  due  to  the  corrective  action  process,  i.e., 
when  the  fixes  are  implemented  and  their  effectivity.  As  pointed  out  earlier,  the  AMPM, 
unlike  a  tracking  model,  is  insensitive  to  any  nonsmoothness  in  the  expected  number  of 
failures  versus  test  time  that  results  from  the  timing  or  effectivity  of  corrective  actions. 
Thus  in  such  a  situation,  program  management  may  wish  to  use  a  projection  method  such 
as  the  AMPM  to  assess  the  reliability  of  the  current  configuration  or  to  project  the 
expected  reliability  at  a  future  milestone. 

As  discussed  in  [3],  the  AMPM  can  also  be  used  to  construct  a  useful  reliability 
maturity  metric.  This  metric  is  the  fraction  of  the  expected  initial  system  B-mode  failure 
intensity,  Ag ,  surfaced  by  test  duration  t.  By  this  we  mean  the  expected  fraction  of  Ag 
due  to  B-modes  surfaced  by  t.  This  concept  will  be  expanded  upon  in  a  later  subsection. 

Prior  to  presenting  the  model  equations  and  estimation  procedures,  we  shall  list 
the  associated  notation  and  assumptions. 

4.4.2  AMPM  Notation  and  Assumptions. 


Notation: 

Mi 


Arithmetic  average  of  the  J,.,  i.e.. 


Parameters  for  gamma  density  function,  where  a>-\  and  /?  >  0 


(subscripted  by  K  or  oo  where  required  for  clarity). 


QO 

a\  Denotes  the  integral  for<3r>-l. 

0 
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A 

V{a,0) 


A 


Gamma  random  variable. 

Moment  generating  function  for  A . 

Denotes  gamma  random  variable  with  parameters  at  >  — 1 ,  /?>0 . 
Denotes  density  function  for  where 

X 


fM) 


for  >?  >  0 ; 


=  0  elsewhere 


A  —  (Aj,-**,A^) 

Random  sample  of  size  K  from  r{a,A}- 

X  =  *  •  *  5  ) 

Realization  of  A . 

^B,K 

K 

Expected  value  of  ^A,  . 

1=1 

A.  = 

//{t  ;  ^ 

Expected  number  of  distinct  B-modes  conditioned 
on  A  =  A- 

h{f.A) 

Expected  rate  of  occurrence  of  B-modes  given 

A  =  A- 

h{t) 

Unconditional  expected  B-mode  rate  of  occurrence 

r{t>A) 

System  failure  intensity  after  fixes  to  B-modes 
surfaced  by  t  have  been  implemented,  conditioned 
on  A  =  A- 

pin  A) 

Expected  value  of  r(r  ;i)  with  respect  to  random 

first  occurrence  times  of  B-modes. 

At) 

Expectation  of  /?(?;  A)  with  respect  to  A. 

iXt) 

Equals  1  if  B-mode  i  occurs  by  t,  equals  0 
otherwise. 
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Failure  intensity  at  time  t  due  to  imsurfaced  B- 
modes,  conditioned  on  A  =  A. 


s{t) 

4) 

ti 


L{m,  t) 

In 

Z 


vk 


obs 


Ko 


Unconditional  expected  failure  intensity  due  to  set 
of  B-modes  surfaced  by  t,  in  absence  of  any  fixes. 

Expected  fraction  of  surfaced  as  a  function 
of  t. 

Time  of  first  occurrence  of  B-mode  i. 


Likelihood  function  for  the  test  data  (m,  t)  given 
A  =  A. 

Expectation  of  L{m,t,^. 

Natural  logarithm  (base  “e”). 
ln{L(m,t)} 

{cc,fi,K) 


Set  of  indices  associated  with  m  observed  B- 
modes. 

Greatest  lower  bound  for  set  of  K-values  for  which 
AMPM  mle’s  are  well  defined. 


K  IBM  model  mle  of  K. 

IBM 

A  Defined  to  be. 


Additional  Assumptions  for  AMPM  -  Continuous 

•  The  time  to  first  occurrence  is  exponentially  distributed  for  each  failure  mode. 
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•  For  i  =  1,2,  •  •  • ,  A^,  the  effectiveness  of  a  fix  associated  with  B-mode  i  is 
independent  of  the  mode’s  initial  rate  of  occurrence  . 

•  The  B-mode  initial  rates  of  occurrence  (/i, ,  •  •  • ,  )  constitute  the  realization  of  a 

random  sample  (A,  , •  •  • ,  )  fi-om  a  gamma  distribution  with  density  .  This 

models  mode-to-mode  variation  in  the  B-mode  initial  failure  rates.  That  is,  we 
assume  the  A,  (z  =  1,  •  •  • ,  A:)  are  independent  and  identically  distributed  (IID) 

random  variables,  where  A,  ~  Y{cx,0). 

4.4.3  AMPM  Development.  The  AMPM  provides  a  procedure  for  assessing  the 
system  failure  intensity  r(r  ;>?) .  Recall  r(r  ;i)  denotes  the  system  failure  intensity  after 
fixes  to  all  B-modes  surfaced  by  test  time  t  have  been  implemented. 

Note  i  =  (/I, ,  •  •  • ,  )  denotes  the  initial  B-mode  rates  of  occurrence.  In 

particular,  consider  B-mode  i.  If  this  mode  does  not  occur  by  t  then  its  rate  of  occurrence 
at  t  is  still  A. .  However,  if  B-mode  i  occurs  by  t  then,  by  our  definition  of  r(r  ;i),  the 

contribution  of  this  mode  to  r{f,A)  is  only  (l  -  due  to  the  implemented  fix  (or 
fixes)  to  mode  i  by  t.  We  may  conveniently  mathematically  express  the  contribution  of 
B-mode  i  to  r{t;^  by 


Thus 

/=1 

=  (2) 

/=!  /=1 

As  in  the  Crow/ AMS AA  model,  the  AMPM  assesses  the  system  failure  intensity  r{t  ;A) 
by  an  assessment  of  the  expected  value  of  r{t;  A),  i.e.  p{t;  A)  =  E{r{f,  A)) .  Note  by  (2) 
we  have 


=  E(r{f,2)) 

=  £(/,(<))  (3) 

/=!  i=\ 


In  Appendix  D,  Annex  1  we  show. 
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£[/,(()]  =  1-e 


(4) 


where  the  expectation  is  with  respect  to  the  time  of  first  occurrence  of  B-mode  i. 
This  yields 


/=l 


(5) 


i=\ 


In  Section  4.3  (where  the  argument  A  was  suppressed)  it  was  noted  that  the 
Crow/AMSAA  model  approximates  p{f,  A)  by 


Pc  i)  =  K  +  Pd  K  i) 


(6) 


1=1 


with 


=  uvt"  '  (7) 

for  positive  constants  u,  v.  This  form  for  the  expected  rate  of  occurrence  of  new  B- 
modes  corresponds  to  approximating  the  expected  number  of  distinct  B-modes  occurring 
over  [0,  t]  by 


Recall  the  Crow/AMSAA  procedure  estimates  the  constants  u,  v  by  the  mle  statistics 
based  on  the  B-mode  first  occurrence  times  observed  during  Test  Phase  I,  i.e.,  [0,  Tj. 
The  summation  term  in  (6)  is  assessed  as 

i^obs 


where  d]  is  the  assessed  fix  effectiveness  factor  for  observed  B-mode  i,  and  A/,-  is  the 

number  of  occurrences  of  failures  during  [0,T]  attributed  to  B-mode  i.  Note  in  the 
Crow/AMSAA  procedure  all  fixes  are  assumed  to  be  delayed  to  the  end  of  the  period 

N- 

[0,T].  Under  this  assumption  is  an  unbiased  estimate  of  A^ .  However,  if  fixes  to  B- 
modes  are  implemented  prior  to  the  end  of  this  period  (9)  may  not  be  an  adequate 

K 

assessment  of  . 

1=1 

The  AMPM  does  not  attempt  to  assess  by  estimating  each  A. .  Instead  the 

AMPM  approach  is  to  view  {A^,---,A^)  asa  realization  of  a  random  sample 
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A  -  (A  ....A,)fromthegammarandomvariaWer(a.^).  Thisallowsoneto 

utilize  all  the  B-mode  times  to  first  occurrence  observed  during 

the  gamma  parameters  a,  /? .  Thus  in  place  of  directly  assessing  yr(r;  i) ,  the  AMPM 

uses  estimates  of  a  and  /?  to  assess  the  expected  value  of  M'lA)  '"'■'ere 


/=1 


(10) 


This  assessed  value  is  then  taken  as  the  AMPM  assessment  of  the  system  failure  mtMsity 
Ifter  fixes  to  all  B-modes  surfaced  over  [O.t]  have  been  tmplemented.  This  approach 
does  away  with  the  need  to  estimate  individual  .  Trying  to  adequately  estimate 

individual  i,  could  be  particularly  difficult  in  the  case  where  many  fixes  are 
implemented  prior  to  the  end  of  the  period  [0,T]. 

From  Equation  (10)  we  see  that  the  expected  value  of  /?(ri  A)  with  respect  to  the 
random  sample  A ,  denoted  by  /^(O ,  is  given  by 


fit)  ^  +2^(l-(i,)£(A,)  +  X^/ 

1=1  '=' 

Recall  the  A,,  are  IID  with  A,.  ~  A .  Thus  £:(A,)  =  E{a)  and  E(A,e  )=  E(Ae  )  for 

i  =  \,-,K  .  After  rearranging  terms  and  replacing  ^^i  ^  ^(A,  )  W  ^(a)s 

E{A,e~^'' )  by  £:(Ae"''')  we  arrive  at 

f,{t)  =  A,Al-Md){KE{A)}^^AKE(^^"‘)} 


Next  note 


■^B,K  ~  E  ^A^ 


=  ke{a) 


(13) 


.  1=1  i 


Thus  we  can  express  p{t)  by 

^,)  =  <'4) 

To  interpret  the  term  K  £(Ae'" )  in  (14)  we  first  note  that  in  Appendix  D, 
Annex  1,  it  is  shown  that 
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i=\ 

Thus  the  expected  rate  of  occurrence  of  new  B-modes  at  t,  given  A ,  is 

h{t;A)  = 

K 

Consider  the  average  (i.e.  expected)  value  of  h{t\  A)  =  ^  A, e  over  all  possible 

1=1 

random  samples  A  =  (A,  ,  •  •  • ,  A ),  where  A,.  ~  A  for  /  =  1, •  •  • ,  .  We  obtain 

E{h{f,K))  =  =  A£(Ae'^')  (15) 

/=! 

Let  h{t)  =  E{h{f,K)).  Thus  h{t)  is  the  unconditional  expected  rate  of  occurrence  of  new 
B-modes  at  test  time  t  averaged  over  all  possible  random  samples  A .  By  (14)  and  (1 5) 
we  have 

pif)  =  (16) 

This  expression  for  p{f)  is  similar  in  form  to  the  Crow/AMSAA  approximation  to 
given  in  Equation  (14): 

Pc  +  X 

/=! 

where  reference  to  A  was  suppressed  in  the  notation. 

The  expression  in  (16)  for  the  expected  system  failure  intensity  after 
incorporation  of  B-mode  fixes  is  actually  quite  appealing  to  one’s  intuition  if  put  in  a 
slightly  different  form.  To  arrive  at  this  form  we  shall  simply  subtract  and  add  the  term 
h{t)  on  the  right  hand  side  of  Equation  (16).  Doing  this  we  can  express  p{t)  by 

p{t)  =  A^+{1-Pj){As^ -h{t)]+h{t)  (17) 

Now  we  see  that  p{t)  is  the  sum  of  three  failure  intensities.  The  first  is  simply  the 
constant  failure  intensity  due  to  the  A-modes.  To  consider  the  second  failure  intensity  we 
shall  first  consider  h{t) .  We  have  shown  that  this  term  is  the  expected  rate  of  occurrence 
of  new  B-modes  at  test  time  t  averaged  over  the  random  samples  A .  Additionally,  h{t) 
is  the  expected  failure  intensity  contribution  to  p{t)  due  to  the  set  of  B-modes  that  have 


d  p{t;^ 


dt 


<=i 
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not  been  surfaced  by  t.  To  see  this,  first  note  that  the  failure  intensity  at  time  t, 
conditioned  on  A  =  i ,  due  to  unsurfaced  B-modes  is  u{t;  i)  where 


/=! 


(18) 


Recall  by  (4), 

with  respect  to  the  first  occurrence  of  B-mode  i.  Thus  by  (18)  we  have 

E[u(f,A)]  =  5^/1, W] 

M  /=l 

=  =  hini)  (19) 

1=1 

It  immediately  follows  from  (19)  that  h{t)  is  the  unconditional  expected  failure  intensity 
due  to  the  set  of  unsurfaced  B-modes  at  time  t,  since  h{t)  =  E{h{f\  A))  . 

Finally,  we  consider  the  second  term  of  f^t)  in  (17).  In  the  absence  of  any  fixes, 
the  sum  of  h(t)  and  the  unconditional  expected  failure  intensity  due  to  the  set  of  B-modes 
surfaced  by  t,  denoted  by  s(t),  must  equal  .  Thus  s{t)  =  Ag  j.  -  h{t) .  If  we 
implement  fixes  to  the  B-modes  surfaced  by  t  with  an  average  FEF  equal  to  fig ,  then  the 
residual  expected  failure  intensity  due  to  the  set  of  surfaced  B-modes  would  be 

(l  -  //rf )  5(1)  =  (1  -  Arf )  {^B.K  -  ^(0} 

In  the  above  equations  we  can  replace  Ag  ,.  by  h(0)  since  at  t=0  all  B-modes  are 
unsurfaced.  Thus 

h{0)  =  Ag^,  (21) 

As  in  Section  4.3,  we  call  the  residual  expected  failure  intensity  approached  by 
p(t)  as  t  tends  towards  infinity  the  growth  potential  failure  intensity,  denoted  by  p^p . 

Since  lim/i(r)  =  0  we  have 

Pcp  = 
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Note  this  expression  has  a  form  similar  to  that  for  the  growth  potential  in  the  Crow/ 
AMSAA  model.  The  quantity  p'^p  is  called  the  growth  potential  MTBF.  The  growth 
potential  for  the  AMPM  is  used  in  the  same  way  as  indicated  in  Section  4.3  for  the  Crow/ 
AMSAA  model. 


Another  useful  quantity  is  the  expected  fraction  of  the  system  expected  initial  B- 
mode  failure  intensity,  Xp ,  surfaced  as  a  function  of  test  time  t.  We  shall  let  Bit)  denote 
this  quantity.  Thus,  by  definition  of  s(t),  we  have 


^B,K 


(23) 


Note  that  9{f)  is  independent  of  the  corrective  action  process.  By  this  we  mean  that  Oif) 
does  not  depend  on  when  fixes  are  implemented  nor  on  how  effective  they  are. 

The  function  9{t)  can  usefully  serve  as  a  measure  of  system  maturity.  Observe 
that  for  a  test  of  duration  t,  no  matter  how  effective  our  fixes  are,  we  can  only  expect  to 
eliminate  at  most  a  fraction  equal  to  9{t)  of  the  expected  B-mode  contribution  to  the 
initial  system  failure  intensity.  Thus  low  values  of  ffif)  would  indicate  additional  testing 
is  required  to  surface  a  set  of  B-modes  that  account  for  a  significant  part  oi  Xg.  A  high 
value  for  d{t)  could  indicate  that  further  testing  is  not  cost  effective.  Resources  would  be 
better  expended  toward  formulating  and  implementing  corrective  actions  for  the  surfaced 
B-modes.  As  part  of  a  reliability  growth  plan  it  would  be  useful  to  specify  goals  for  Bit) 
at  several  program  milestones. 

Next  we  shall  express  the  key  AMPM  reliability  projection  quantities  in  terms  of 
K  and  the  gamma  parameters  a  and  P .  By  Appendix  D,  Annex  2,  we  have 


=  Kj3{a  +  \) 

(24) 

II 

\ 

+ 

+ 

(25) 

h{t) 

Kp{a  +  \)  __  dpit) 

(26) 

+  ’  dt 

pit) 

=  i.  +  (1  -  )Kfi(a  + 1)  +  ^ 

(27) 

and 

B{t)  =  +  (28) 
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Utilizing  Equation  (24)  for  /Ig  ^ 


we  can  also  express  h{t)  and  p{t)  as  follows. 


h{t) 


^B.K 


(29) 


and 


M')  =  ^  +  (l  -  Aj  ^ 


(30) 


In  .he  next  section  we  shall  consider  the  behavior  of  the  AMPM  as  K  incre^es. 

I  imitinn  etr^ssToS  tire  AMPM  quantities  in  (24)  through  (30)  will  be  obtained  as 
Limiting  expr  onri/?— /?  Then  parameter  estimation 

— >  00  under  natural  assumptions  about  Xg  j.  and  p  p^  ■ 

procedures  will  be  specified  for  the  finite  K  AMPM  and  the  limiting  parameters  as 
a:-»co. 

4  4  4  Limiting  Behavior  of  AMPM.  We  shall  now  consider  the  limiting  behavior 
of  the'^lPM  as  K  increases.  To  do  so  we  first  define  step  processes 

1=1, where 

[  1  if  B-  mode  i  occurs  by  t 
~  |q  otherwise 


Note 


Pr(^x,,(0^2)  =  0 


(31) 


and 


Pr(A-x„W=l)  =  1-Pr(2fr,,(')  =  0)  (32) 

need  only  specify  Pr(.lfK.,(t)=o). 


we 


Thus  to  complete  our  definition  of  these  processes 

To  keep  the  definition  of  these  processes  consistent  with  the  AMPM  assumptions  we 

(33) 


define 


Pr(^K,,  (0  =  O)  =  f^ 


where  A  ~  r(flr  B)  and  is  the  previously  defined  gamma  density  function  with 

r=..  and  /?=!.  Note  is  the  unconditional  AMPM  indicator  function  for  B 

mode  i  corresponding  to  the  earlier  defined  conditional  indicator  function  /,  (0  where 
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Pr(/,(l)  =  0)  = 


and  subscript  K  was  suppressed.  By  (33)  and  Appendix  D,  Annex  2 

Pr(A^,(r)  =  0)  =  E[e-‘^)  =  =  {\  + (34) 

From  (32)  and  (34)  we  obtain 

Mk^)  a  E 

- 

=  tMx,A‘)=i)  =  =  A‘)  (35) 

/=1 

for  {a,.  ,J^k)  =  AMPM  step  processes  (a  (r),  0  <  r  <  oo},  l<i<K , 

give  rise  to  our  previously  developed  AMPM. 

To  investigate  the  behavior  of  our  projection  model  as  K  increases,  we  must 
specify  the  limiting  behavior  of  aj^.  and  .  Since  is  simply  a  scale  factor  for  test 
time  t  it  is  reasonable  to  keep  /3f^  fixed,  sdcy  &  (0»°°)  •  Recall  by  (24), 

A.g^  =K  +l)-  Regardless  of  the  value  of  K,  Agj^  represents  the  unconditional 

expected  B-mode  contribution  to  the  initial  system  failure  intensity.  Thus  it  is  natural  to 
let  K  Pf.  + 1)  =  ^s,»  e  (O.  °o)  for  K-  Actually,  to  obtain  our  results  for  the 

limiting  behavior  of  the  AMPM  we  need  only  insist  that 

limy^^  =  P„e(0,co)  (36) 

K^oo 

and 

lim  K  {a^  +  0  =  ^  (0>  (37) 

/T-^oo 

We  shall  simply  denote  P^  and  Xg^  by  /?  and  ig ,  respectively.  Since  + 1  >  0 ,  (36) 
and  (37)  imply 

limor^  =  -1  (38) 

/C->oo 

Let  (r)  be  the  supposition  of  the  independent  step  processes  X^^it),  i.e. 
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(39) 


I=l 

„  is  demonstrated  in  [4]  that  the  stochastic  process  {X,  (t),  O  S  r  <  »} 
nonhomogeneous  Poisson  process  (NHPP)  with  mean  value  function  M.  (<) 

where 


A.W  =  + 


(40) 


This  result  suggests  that  for  complex  x^{t)  is  the 

cc  iv  (A  n</<ool  to  behave  like  aNHPP 

nTmLr  of  distinct  B-modes  that  occur  by  t  and  £{X.(r))  =  A.(<)  8"'®"  ” 

we  can  now  relate  the  hey  AMPM  reliaWhy  pr^^tion  quamUies  -  (24)  .hough 

1S"m  <,u«  NHPP  quantities  by  « .  Thus,  for  example, 

by  (24)  and  limit  condition  (37)  we  have 


lim 

/C->oo 


(0,co) 


(where  we  also  denote  simply  by  A, ).  By  (26)  we  also  have 


‘cK/^A^k  +0 


rx  _  VA  yzK  ^ 


dz 


(41) 


Thus 


i 

lim  //jf  (0  =  II 


K-^co 


lini  ^  "T  \arjif  +2~  I 

'^-“[(l  +  /?^z) 


dz 


By  (36)  through  (38)  and  (40)  this  yields 

/  \  r 


v/?y 


ln(l  +  y^0  = 


Again  by  (26),  (36)  through  (38)  and  (40),  we  obtain 

hmft.W  - 


(42) 
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(43) 


_A_ 


=  Kit) 


By  (27),  (36)  through  (38)  and  (43)  we  arrive  at 


f^d  ^  Pk^k  O] 

a^*Z 


lim^,(0  =  lim  1^,  +  (1  -  //.  A  K  + 1)  + 


Additionally,  by  (22)  and  (41 )  we  have 

lim  Pcp,K  =  1™  {^A  +  (l  -  Arf  )  } 

K^OO  A->0O 

=  +  (l -//j  )/lfl  =  Pgp,x 


(44) 


(45) 


Finally,  by  (28),  (36)  and  (38)  we  deduce 

lim  0^  (0  =  lim  J  -  (l  +  A  0  } 

AT ->00  A-><» 


Pt 

\  +  pt 


Thus  by  (43)  we  conclude 

lim  (0 

/C^QO 


Pt 

1  +  pt 


i, -*.(>) 

Ag 


0.{t)  (46) 


4,4.5  Estimation  Procedure  for  AMPM.  In  this  section  we  shall  specify  the 
procedmes  to  estimate  key  AMPM  parameters  and  reliability  measures  expressed  in 
terms  of  these  parameters.  Estimation  equations  will  be  given  for  the  finite  K  and  NHPP 
variants  of  the  continuous  AMPM.  The  model  parameter  estimators  are  mle’s.  Statistical 
details  and  further  discussion  of  the  estimation  procedures  are  provided  in  Appendix  D, 
Annex  3. 

Our  parameter  estimates  are  written  in  terms  of  the  following  data:  m  =  number  of 
distinct  B-modes  that  occur  over  a  test  period  of  length  T,  t  =  (r,  ,•••,/„ )  where 
Q  <  <T  are  the  first  occurrence  times  of  the  tn  observed  B-modes,  and 

=  number  of  A-mode  failures  that  occur  over  test  period  T.  We  shall  denote  an 
estimate  of  a  model  parameter  or  expression  by  placing  the  symbol  over  the  quantity. 


124 


The  finite  K  AMPM  estimates  are  based  on  a  specified  value  of  K.  If  we  hold  the 
test  data  constant  and  let  K-^oo  we  obtain  AMPM  projection  estimates  that  are 
appropriate  for  complex  subsystems  or  systems  that  typically  have  many  potential  B- 
modes.  The  AMPM  limit  estimating  equations  are  derived  in  Appendix  D,  Annex  3. 
These  equations  can  also  be  obtained  from  mle  equations  for  the  NHPP  associated  with 
the  AMPM.  This  process  was  discussed  in  Section  4.4.4  and  has  the  mean  value  function 

given  by  Equation  (40). 

Recall  af. ,  are  the  gamma  parameters  for  the  AMPM  where  it  is  assumed  the  K 
initial  B-mode  failure  rates  are  realized  values  of  a  random  sample  from  a  gamma 
random  variable  r{af. ,Pk)- 


The  mle  for  Pf.  is  p^  where 


K  = 


-  ^ 

(  "  ^ 

t 

mPf, 

2.  In  . 

/  .1  ^ 

I'-'  1  + 

1'  =  '  1  +  /^K  h  j 

T-t, 

'=■  1  +  P,  /, 


(47) 


In  1  +  P^T 


1 


S- 


\  +  pT 


The  mle  for  Is  a  k  where  Ok  can  be  easily  obtained  from  and  either  equation 
below.  These  equations  are  the  maximum  likelihood  equations  for  a,,  and  P^ 
respectively  (see  Appendix  D,  Annex  3); 


|^(3r/c  +  l 


-1 


m 


^In 


A  ^  m 

1+/’,?'  -S'” 


(  "  ^ 


1  ^ 


yj 


(48) 


aK-t-l 


P.  + 

[K-m)T  t _ 

\  +  P,T  '='l  +  ^^t, 


(49) 
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we  can  estimate  all  our  finite  K  AMPM  quantities  where  the 


Using 

A-mode  failure  rate  is  estimated  by  and  the  average  B-mode  fix 

effectiveness  factor  //^  is  assessed  as 


m  ieobs 


(50) 


In  (50),  the  assessment  d*  of  the  fix  effectiveness  factor  (FEF)  for  observed  B-mode  i 

will  often  be  based  largely  on  engineering  judgement.  The  value  of  should  reflect 

several  considerations:  (1)  How  certain  we  are  that  the  problem  has  been  correctly 
identified;  (2)  the  nature  of  the  fix,  e.g.,  its  complexity;  (3)  past  FEF  experience  and  (4) 
any  germane  testing  (including  assembly  level  testing). 

Note  the  left-hand  side  of  Equation  (47)  requires  a  value  for  K  before  we  can 

A 

numerically  solve  for  In  practice  we  do  not  know  the  value  of  K.  We  could  attempt 

to  use  the  data  {m,t)  to  statistically  estimate  K.  However,  graphs  presented  in  the  next 
section  illustrate  the  difficulty  in  obtaining  a  reasonable  estimate  for  K  even  for  a  large 
data  set  that  appears  to  fit  the  model  well.  Thus  we  prefer  to  take  the  point  of  view  that 
we  should  not  attempt  to  statistically  assess  K.  However,  by  conducting  a  standard 
failure  modes  and  effects  criticality  analysis  (FMECA),  we  can  place  a  lower  bound  on 
K,  say  K ( .  Our  experience  with  the  AMPM  indicates  that  if  K  is  substantially  higher 
than  m  ,  say,  e.g.,  AT  >  10m ,  then  our  AMPM  projection  quantities  will  be  insensitive  to 
the  value  of  K.  We  believe  for  a  complex  system  or  subsystem  it  will  often  be  the  case 
that  Kf  >  10m  or  at  least  the  unknown  value  of  K  will  be  10m  or  higher.  The  factor  10 

may  be  larger  than  necessary.  We  suggest  exercising  the  finite  K  AMPM  with  several 
plausible  lower  bound  values  for  K  and  comparing  the  associated  projections  with  those 
obtained  in  the  limit  as  AT  — >•  co  .  This  is  illustrated  for  a  data  set  in  the  next  section. 

consider  the  sequence  of 

satisfies  Equation  (47) 


To  obtain  the  limiting  AMPM  projection  model  estimates 

finite  K  AMPM  estimates  (j0k)  ^  assume 

\  I  K>Ko 

for  each  K>K^.  In  Appendix  D,  Annex  3  it  is  shown  that 


=  lim>^^  €(0,oo) 

A  — ►«) 

is  a  finite  positive  value.  Moreover,  it  is  demonstrated  that 


(51) 


126 


f  ^  M  «  1  mj^^T 

In  l  +  /^.r  Z - ^ - : - 

^  ■'>•■'1  +  ^,1.  \  +  /}.T 


It  is  also  shown  that 


or*  =  MraaK  =  -1 


where  for  each  p^,K>K^,aK  satisfies  Equation  (48)  (or  Equation  (49)).  The 

A  /V 

limiting  AMPM  estimates  /?„  and  /I* ,  given  below  in  Equation  (55),  can  be  shown  to 
be  mle’s  for  parameters  and  .  Recall  these  parameters  define  the  NHPP 
discussed  in  Section  4.4.4  whose  mean  value  function  is  given  in  Equation  (40). 

For  ease  of  reference,  the  finite  K  AMPM  and  limiting  AMPM  estimates  for  key 
projection  model  quantities  are  listed  below  and  indexed  by  K  and  oo,  respectively: 


Xk  = 


In 


Mk^)  =  ^1- 


M.{')  =  ^  in 

U.  J  ^ 


hxit)  = 


l  +  Pict 


ho={t)  = 
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PgP,K  + 

A  A  ^ 

(60) 

PgP.oo  = 

(61) 

“  Pgp,k'^  Md 

A  A  A 

(62) 

/’=o(0  =  /^c;p,oo  +  A>»(0 

(63) 

0K{t)  =  1-  l  +  ^^tj 

A 

(64) 

II 

8 

< 

(65) 

Note  (55)  together  with  (57)  imply 


A 


Xt)  = 


r  -  ^ 

ff,00 


In 


=  m 


(66) 


This  agrees  with  intuition  in  the  sense  that  (t)  is  an  estimate  of  the  expected  number 

of  distinct  B-modes  generated  over  the  test  period  [0,T]  while  m  is  the  observed  number 
of  distinct  B-modes  that  occur. 

Suppose  we  adopt  the  view  that  our  “model  of  reality”  for  a  system  or  subsystem  is 
the  AMPM  for  a  finite  K  which  is  large  but  unknown.  Then  we  can  consider  the  limiting 
AMPM  projection  estimates  as  approximations  to  the  AMPM  estimates  that  correspond 
to  the  “true”  value  of  K.  Our  discussion  in  this  section  suggests  that  over  the  projection 
range  oi  t>T  values  of  practical  interest,  the  limiting  estimates  should  be  good 
approximations  for  complex  systems  or  subsystems.  In  this  sense,  knowing  the  “true” 
value  of  K  is  usually  unimportant.  Note,  however,  it  is  useful  to  have  available  the 
computational  formulas  for  the  finite  K  AMPM  projection  estimators  as  a  function  of  K. 

For  example,  we  can  compare  the  graphs  of  a  projection  estimator  such  as  (t)  or 

A 

(r)  over  the  t  range  of  interest  for  different  values  of  K  to  the  corresponding  limiting 
estimator.  In  this  fashion  we  can  discern  the  nature  of  the  convergence,  for  example,  the 
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rapidity  of  convergence  and  whether  the  convergence  is  strictly  increasing  or  decreasing 
for  t  values  of  interest.  This  type  of  graphical  analysis  is  illustrated  with  an  example. 

4.4.6  Example.  We  shall  illustrate  several  key  features  of  our  projection  model 
and  associated  estimators  by  applying  the  model  to  a  data  set  generated  during  an  Army 

A 

system  development  program.  Here,  we  shall  just  focus  on  the  B-modes  and  let  A  a  =  0. 
This  test  data  set  consists  of  w  =  163  B-mode  first  occurrence  times  generated  over 
T  =  8000  “equivalent”  mission  hours. 

In  Figure  1,  we  display  the  cumulative  number  of  distinct  B-modes  versus  the 

mission  hours.  We  also  display  the  graphs  of  {t)  for  several  values  of  K.  We  can 
show  that  the  greatest  lower  bound,  ,  for  the  set  of  K-values  for  which  the  AMPM 
estimators  are  well  defined  corresponds  to  a  degenerate  gamma.  This  limiting  gamma 
density  has  zero  variance  and  mean  equal  to  X ,  where  A;  =  X  for  i  =  l,---,K  .  To  avoid 
numerical  instability,  separate  maximum  likelihood  equations  were  derived  and  used  for 
this  limiting  case.  On  our  graphs  we  have  labeled  the  curves  associated  with  this  case 
(i.e.,  K  =  A^^o)  IBM  to  indicate  that  this  limiting  form  for  //(t)  coincides  with  the  IBM 

model  [5].  More  explicitly,  the  IBM  model  uses  this  //(t)  for  the  expected  number  of 
“non-random”  failures  experienced  in  t  test  hours.  This  limiting  form  for  //(r)  also  is 
used  by  Musa  in  his  software  reliability  basic  execution  time  model  [6].  It  is  interesting 
to  note  that  the  opposite  AMPM  limiting  form,  is  used  by  Musa  and  Okumoto  in 

their  Logarithmic  Poisson  software  reliability  execution  time  model  [7].  In  both  of 
Musa’s  models,  //(t)  represents  the  expected  number  of  software  failures  expenenced 

over  test  period  [0,t],  where  t  denotes  execution  time. 

Note  over  the  data  range,  i.e.,  0  <  t  <  8000  hours,  the  graphs  of  (t)  are  visually 
indistinguishable  for  K„^<K<^.  In  such  circumstances  the  value  of  K  cannot  be 
reasonably  assessed  from  the  test  data  even  if  one  can  formally  obtain  an  mle  for  K.  In 
fact,  applying  the  IBM  model  (where  all  X.,  are  implicitly  assumed  to  be  equal),  we  can 

always  obtain  an  mle  for  K  whenever 


(see  Musa,  lannino,  and  Okumoto  with  respect  to  the  exponential  class  family  [8]). 
However,  it  has  been  our  experience  that  the  IBM  estimate  of  K,  =  ATg ,  is  often 
only  marginally  higher  than  m  ,  the  observed  number  of  distinct  B-modes.  Since  {t) 
approaches  K  as  t  -» <» ,  such  a  low  estimate  of  K  forces  the  slope  of  (t)  to  quickly 
approach  zero  beyond  T  (Figure  2).  Note  {t)  is  the  slope  of  Mk  (0-  Thus  we  can  see 

that  such  a  low  estimate  of  K  quickly  forces  Aat  (t)  close  to  zero  fox  t>T .  This  m  turn 
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tends  to  produce  an  “optimistic”  failure  intensity  projection,  especially  when  the  assessed 
value  of  pd  is  high.  This  follows  from  the  formula 

Pk^)  -  y^A  +  il- jUj)Ab,k  + hK{t)  (67) 

which  applies  for  <K<oo.  Thus  a  good  fit  over  [0,T]  is  not  a  sufficient  condition 
to  ensure  that  a  projection  model  will  provide  reasonable  projection  estimates  for  t^T. 

Looking  at  Figure  3,  as  one  might  expect,  the  model  with  K  =  <x>  appears  to 
provide  a  more  conservative  estimate  of  p^it)  for  t>T  than  do  the  finite  K  estimators. 

However,  for  t  >  T ,  it  is  important  to  note  that  the  (t),  (f)  and  <9^  (t)  graphs, 

displayed  in  Figures  2,  3,  and  4,  respectively,  quickly  become  much  closer  to  the 
corresponding  K  =  oo  graph  than  to  the  K  =  K,s^  graph  as  K  increases  above  . 

Observe  from  Figure  4,  0^. (SOOO) « .67  for  <K  <co .  Thus,  whatever  the 
“true”  value  of  K,  we  estimate  that  the  remaining  B-modes  contribute  about  33  Ag  to  the 
system  failure  intensity. 
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Cumulative  Number  of  B-modes 


Mission  Hours 


Figure  1.  Observed  Versus  Estimate  of  Expected  Number  of  B-Modes. 


Mission  Hours 


Figure  2.  Extrapolation  of  Estimated  Expected  Number  of  B-Modes  As 

Function  of  K. 

(Data  Ends  at  8000  Hours) 
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Projected  MTBF  (Hours) 


Mission  Hours 


Figure  3.  Projected  MTBF  for  Different  K’s. 
(Based  on  Initial  8000  Hours  of  Data) 


Mission  Hours 


Figure  4.  Estimated  Fraction  of  Expected  Initial  B-Mode  Failure 
Intensity  Surfaced  for  Different  K’s. 

(Based  on  Initial  8000  Hours  of  Data) 
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^  Before  going  into  the  specifics  of  devising  reliability  growth  planning  curves,  it  is 
useful  to  look  at  the  history  of  this  process  to  learn  why  the  curves  have  the  form  that 
they  do. 

The  earliest  reference  that  we  have  found  on  this  subject  is  An  Analytical  Model 
nf  Reliability  Growth  Through  Testing  by  H.  K.  Weiss,  Handbook  No.  54  304,  Hp,  AD- 
035  767,  May  1954,  Northrop  Aircraft  Inc.,  Hawthrone,  California.  Also,  a  useful  survey 
of  some  early  reliability  growth  methods  is  Reliability  Growth  Modeling  by  Larry  H. 
Crow,  Technical  Handbook  No.  55,  August  1972,  U.S.  Army  Materiel  Systems  Analysis 
Agency,  Aberdeen  Proving  Ground,  Maryland. 

The  Duane  Postulate. 

James  T.  Duane,  an  engineer  with  General  Electric’s  Motor  and  Generator 
Department,  published  a  paper  titled  “Learning  Curve  Approach  to  Reliability 
Monitoring”  in  TFF.F,  Transactions  on  Aerospace,  Vol.  2,  No.  2, 1964.  This  paper 
recorded  his  observation  that  if  changes  to  improve  reliability  (which  are  now  termed 
fixes)  are  incorporated  into  the  design  of  a  system  under  development,  then  on  a  log-log 
plot  the  graph  of  cumulative  failure  rate  vs.  cumulative  test  time  is  linear.  This 
observation  has  become  known  as  the  “Duane  Postulate.”  This  empirically  denved 
statement  is  the  key  to  the  most  commonly  accepted  growth  model  in  use  today  (see 
Section  6. 1 .4).  A  graph  given  in  Duane’s  paper  is  shown  in  Figure  6.1 .  The  straight  lines 
are  based  on  a  least  squares  fit  of  the  data.  The  negative  slope  of  each  line  is  defined  to 
be  the  growth  rate,  a,  for  that  line. 

Duane’s  Growth  Model. 


On  a  Log-Log  Plot,  the  graph  of 

Cumulative  Failure  Rate 
Vs 

Cumulative  Test  Time 
is  Linear 


Let  N  (t)  =  the  total  Number  of  failures  by  time  t.  Then  the  average  failure  rate, 
also  called  Cumulative  failure  rate  C(t),  can  be  found  by  dividing  N(t)  by  t. 


Let  S  be  the  y-intercept  on  a  log-log  plot  of  the  straight  line  that  Duane 
postulated.  The  slope-intercept  formula  for  this  line  then  becomes. 

LogC{t)  =  S-aLogt 

where  log  denotes  the  natural  (base  e).logarithm  (although  any  base  could  be  used). 

The  Duane  Postulate: 

LogC(t)  =  aLogt 


Taking  anti-logs 


C(0  =  /if 


where 


S  =  \rA 

Multiplying  C(t)  by  t  gives  N(t),  and  multiplying  by  t  adds  1  to  the  exponent,  t 


1-ar 


So 


N{t)  =  it 


\-a 


Taking  the  first  derivative  of  the  number  of  failures  with  respect  to  time  gives  the 
instantaneous  failure  rate,  r(t),  at  time  t. 


r{t) 


dN{t) 
d  t 


Z{\-a)t-^ 


Duane’s  model  thus  has  two  parameters,  a  and  i.  The  first,  a,  determines  the 

shape  ofthe  growth  curve.  The  second,  .1,  is  the  size  parameter  for  the  cune  With 

tee  two  paLeters,  the  cumulative  number  of  failures  N(t),  the  average  faitee  rate 
cm  tL  instantaneous  failure  rate  r(t)  can  be  calculated  for  any  time  t  within  th  test 
Further  given  rr  and  A,  it  is  possible  to  solve  for  t,  the  amount  of  testing  tune  it  will  ti*e 
to  actee  a  specific  reliability.  This  assumes  that  the  factors  affecting  reliability  growth 
remain  unchanged  across  the  development. 

F)rawbaclcs  to  Duane^s  Nlethod.  j  4.  u 

Duane  stated  that  ar  could  be  universally  treated  as  being  .5  as  that  seemed  to  be 

the  modal  value  within  his  database.  This  has  since  been  shown  to  be  unrealistic.  It  does 
not  allow  for  different  test  environments  causing  failures  to  be  surfaced  at  different  rates, 
and  for  different  levels  of  engineering  effort  causing  different  rates  of  fix  insertion. 
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The  reliability  values  calculated  using  his  method  are  treated  as  being 
deterministic.  That  is,  there  is  no  allowance  for  the  variation  that  is  typically  observed 
about  an  estimated  value,  and  there  is  no  way  of  judging  whether  the  observed  value, 
which  rarely  matches  the  estimated  value,  is  close  enough.  Further,  there  is  no  way  to 
check  whether  the  model  is  valid  for  the  current  test  situation. 

All  Duane  growth  curves  pass  through  the  origin  of  the  graph.  That  is,  the  item 
under  test  is  imputed  to  have  zero  reliability  at  the  start  of  test. 

The  Crow/AMSAA  Growth  Model. 

Larry  H.  Crow  while  at  the  U.S.  Army  Materiel  Systems  Analysis  Activity’s 
Reliability  and  Maintainability  Division  published  Reliability  Analysis  for  Complex. 
Repairable  Systems.  Technical  Handbook  No.  138,  December  1975,  U.S.  AMSAA, 
Aberdeen  Proving  Ground,  Maryland.  In  this  report.  Dr.  Crow  explored  the  advantages 
of  using  a  Nonhomogeneous  Poisson  Process  with  a  Weibull  intensity  function  to  model 
several  phenomena,  including  reliability  growth.  If  system  failure  times  follow  the 
Duane  Postulate,  then  they  can  be  modeled  as  a  Nonhomogeneous  Poisson  Process  with 
Weibull  intensity  function.  To  make  the  transition  from  Duane’s  formulae  to  the  Weibull 
intensity  functional  forms,  p  has  to  be  substituted  for  \  -a.  Thus  the  parameters  in  the 
Crow  model  are  X  and  p  where  P  determines  the  shape  of  the  curve.  The  physical 
interpretation  of  P  (called  the  growth  parameter)  is  the  ratio  of  the  current 
(instantaneous)  MTBF  to  average  (cumulative)  MTBF  at  time  t. 

This  stochastic  interpretation  immediately  brings  the  benefits  of  Statistics  to  the 
formulae  that  Duane  had  derived.  That  is,  the  parameters  X  and  f)  can  be  determined 
using  maximum  likelihood  estimators  (mle’s)  rather  than  f)  being  assumed  to  be  fixed. 
Further,  hypothesis  tests  and  confidence  limits  can  be  determined  for  the  parameters,  and 
Goodness-of-Fit  tests  can  be  performed  on  the  model.  This  eliminates  the  first  two 
drawbacks  of  Duane’s  model.  We  will  discuss  later  how  Crow  handles  the  problem  of 
imputed  zero  reliability  at  the  start  of  test. 

One  should  take  note  that  even  though  the  growth  rate  estimate  a  can  be  calculated  from 
Crow’s  growth  parameter  estimate, and  it  is  still  interpreted  as  the  estimate  of  the 
negative  slope  of  a  straight  line  on  a  Log-Log  plot.  Crow’s  estimates  of  X  and  fi  are 
somewhat  different  from  the  ones  derived  using  Duane’s  procedures.  This  follows  from 
the  fact  that  the  estimation  procedure  is  mle,  not  least  squares,  thus  each  model’s 
parameters  correspond  to  different  straight  lines,  respectively. 
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TABLES 


The  following  tables  provide  approximations  to  the  probability 
ProbU-  u  d)  where  a  denotes  the  expected  number  of  failures  and  d  -  M(T)/TR,  The 
Into  entries  were  cafcnlated  using  a  modification  to  Equation  (25).  TOis  modification 
.miMtapproximating  <(n)  by  and  (2)  conditioning  on  Ai.2  instead  of 

A  >  1 .  Thus,  in  Equation  (25)  the  expression  1  -e'^  is  replaced  by 
1  -  P(N  <  1)  =  I  -  e'"'  -  and  the  summation  is  over  N>2. 

The  approximation  used  for  zj(n)  follows  from  the  lower  confidence  bound 
approximation  given  by 


^(n,s)  s  (n/Zn42.r)^n 


(1) 


where  M  is  the  mle  of  M(T)  calculated  from  the  observed  data  s  -  (t,.  t2....t.).  Here  r, 

denotes  th'e  cumulative  operating  time  to  the  failure. 

TxSSn  e  LTditTprSaL  in  (1)  results  in  slightly  more  conservative  lower 

M(r)  A(n.  s).  This  implies  that  use  of  the  corresponding  apP— n 

to  zHn)  would  yield  slightly  smaller  valuesofProb(^;A.<f)  than  one  would  obtain  by 
utihring  zHn).  Based  on  our  experience  with  Prob(AA.<f)  estimated  by  simulation,  the 

.  1  tn  hp  within  0  0 1  of  values  obtained  through  simulation. 

rrorJ=a;"iU^^ 

between  the  lower  confidence  bound  approximation  given  by  (1)  and  <he  lo 
confidence  bound  using  zj(n)  was  based  on  Table  2  contained  in  Section  3.  Since 

entries  in  this  table  were  for  «  2  2 .  the  probability  of  acceptance,  Ptob(d;yr,  d),  was 

conditioned  on  iV  S  2 .  In  most  cases  of  interest  for  *c  “odeU'S^u^^^^  ‘wcU  valJes 
Prob(W  2  2)  will  be  close  to  one.  In  this  situation,  conditioning  on  AT  a  2  yiel^  values 

of  Pmb(d;  f^.d)  that  are,  for  practical  purposes,  essentially  the  same  as  those  obtained  by 
conditioning  on  N  >  I- 

The  entries  in  these  tables  were  calculated  using  the  well-known  relationship 
between  the  complement  of  a  Chi-square  distribution  function  and  the  cumulative 
Poisson  sum.  This  relationship  was  applied  to  calculate 


Prob 


'  2  > 
X2n  ~ 


I 


2pd 


in  the  expression  for  Prob(d;A,rf)  in  Section  2.1.3  with  zj  (n)  replaced  by  its 
approximation,  i.e.,  W..,.,  ■  In  terms  of  the  cumulative  Poisson  sum,  this  yields 
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Prob 


-w 


(2) 


2^2  iy^Zn+2,r  ) 


jud 


n-l 

=  1  = 

x=0 


w 

x! 


where 

W  = 

With  additional  computational  effort,  one  can  more  precisely  calculate  ?roh{A,/j,d)  by 
iteratively  solving  for  z,(n)  as  the  z-solution  to  Equation  (13)  of  Section  2.1.3  over  an 
appropriate  range  of  n.  Then  Equation  (2)  can  be  utilized  with  2nzUr  replaced  by 

z^(«)/2 . 

The  tables  contained  in  this  appendix  are  approximation  values  of  ?roh{A;/j,d) 
for  three  confidence  levels;  namely,  for  y  =  0.70,  y  =  0.80,  and  y  =  0.90. 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  70  PERCENT  CONFIDENCE 


EXPECTED  NUMBER  OF  FAILURES 


M(T)/TR 

5 

6 

7 

8 

9 

10 

11 

12 

1.00 

0.131 

0.150 

0.163 

0.173 

0.180 

0.186 

0.191 

0.195 

1.05 

0.150 

0.171 

0.187 

0.199 

0.208 

0.216 

0.224 

0.230 

1.10 

0.169 

0.194 

0.212 

0.226 

0.238 

0.249 

0.258 

0.266 

1.15 

0.189 

0.217 

0.238 

0.255 

0.269 

0.282 

0.294 

0.304 

1.20 

0.209 

0.240 

0.264 

0.284 

0.301 

0.316 

0.330 

0.343 

1.25 

0.231 

0.264 

0.291 

0.314 

0.334 

0.351 

0.368 

0.383 

1.30 

0.252 

0.289 

0.319 

0.344 

0.367 

0.387 

0.405 

0.422 

1.35 

0.274 

0.314 

0.347 

0.375 

0.400 

0.422 

0.443 

0.462 

1.40 

0.296 

0.339 

0.375 

0.405 

0.432 

0.457 

0.479 

0.500 

1.45 

0.318 

0.364 

0.402 

0.435 

0.465 

0.491 

0.515 

0.538 

1.50 

0.340 

0.389 

0.430 

0.465 

0.496 

0.525 

0.550 

0.574 

1.55 

0.362 

0.414 

0.457 

0.494 

0.527 

0.557 

0.584 

0.609 

1.60 

0.384 

0.438 

0.484 

0.523 

0.557 

0.588 

0.616 

0.642 

1.65 

0.406 

0.462 

0.510 

0.550 

0.586 

0.618 

0.647 

0.673 

1.70 

0.427 

0.486 

0.535 

0.577 

0.614 

0.647 

0.676 

0.703 

1.75 

0.448 

0.509 

0.560 

0.603 

0.641 

0.674 

0.704 

0.730 

1.80 

0.469 

0.531 

0.583 

0.628 

0.666 

0.700 

0.729 

0.756 

1.85 

0.489 

0.553 

0.606 

0.651 

0.690 

0.724 

0.754 

0.780 

1.90 

0.509 

0.575 

0.628 

0.674 

0.713 

0.746 

0.776 

0.802 

1.95 

0.529 

0.595 

0.650 

0.695 

0.734 

0.768 

0.797 

0.822 

2.00 

0.548 

0.615 

0.670 

0.716 

0.754 

0.787 

0.816 

0.840 

2.05 

0.566 

0.634 

0.689 

0.735 

0.773 

0.806 

0.833 

0.857 

2.10 

0.584 

0.652 

0.708 

0.753 

0.791 

0.823 

0.849 

0.872 

2.15 

0.601 

0.670 

0.725 

0.770 

0.807 

0.838 

0.864 

0.885 

2.20 

0.618 

0.687 

0.742 

0.786 

0.823 

0.853 

0.877 

0.898 

2.25 

0.634 

0.703 

0.758 

0.802 

0.837 

0.866 

0.890 

0.909 

2.30 

0.650 

0.719 

0.773 

0.816 

0.850 

0.878 

0.901 

0.919 

•2.35 

0.665 

0.733 

0.787 

0.829 

0.863 

0.889 

0.911 

0.928 

2.40 

0.679 

0.747 

0.800 

0.841 

0.874 

0.900 

0.920 

0.936 

2.45 

0.693 

0.761 

0.813 

0.853 

0.884 

0.909 

0.928 

0.943 

2.50 

0.706 

0.774 

0.825 

0.864 

0.894 

0.917 

0.936 

0.950 

2.55 

0.719 

0.786 

0.836 

0.874 

0.903 

0.925 

0.942 

0.955 

2.60 

0.732 

0.797 

0.846 

0.883 

0.911 

0.932 

0.948 

0.960 

2.65 

0.743 

0.808 

0.856 

0.892 

0.918 

0.938 

0.954 

0.965 

2.70 

0.755 

0.818 

0.865 

0.900 

0.925 

0.944 

0.958 

0.969 

2.75 

0.766 

0.828 

0.874 

0.907 

0.932 

0.950 

0.963 

0.973 

2.80 

0.776 

0.837 

0.882 

0.914 

0.937 

0.954 

0.967 

0.976 

2.85 

0.786 

0.846 

0.889 

0.920 

0.943 

0.959 

0.970 

0.978 

2.90 

0.795 

0.855 

0.896 

0.926 

0.947 

0.963 

0.973 

0.981 

2.95 

0.804 

0.862 

0.903 

0.932 

0.952 

0.966 

0.976 

0.983 

3.00 

0.813 

0.870 

0.909 

0.937 

0.956 

0.969 

0.979 

0.985 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  70  PERCENT  CONFIDENCE 

EXPECTED  NUMBER  OF  FAILURES 


M(T)/TR 


29 


30 


31 


32 


33 


34  35  36 


LOO 

1.05 

1.10 

1.15 
1.20 

1.25 
1.30 

1.35 

1.40 

1.45 

1.50 

1.55 
1.60 

1.65 

1.70 

1.75 
1.80 

1.85 

1.90 

1.95 
2.00 
2.05 
2.10 

2.15 
2.20 

2.25 
230 

2.35 

2.40 

2.45 

2.50 

2.55 
2.60 

2.65 

2.70 

2.75 
2.80 

2.85 

2.90 

2.95 
3.00 


0.231 

0.232 

0.233 

0.234 

0.291 

0.293 

0.295 

0.297 

0.355 

0.359 

0.362 

0.366 

0.422 

0.427 

0.432 

0.437 

0.489 

0.495 

0.501 

0.507 

0.554 

0.562 

0.569 

0.576 

0.616 

0.624 

0.632 

0.640 

0.674 

0.683 

0.691 

0.699 

0.727 

0.735 

0.744 

0.752 

0.773 

0.782 

0.790 

0.798 

0.814 

0.822 

0.830 

0.838 

0.849 

0.857 

0.864 

0.871 

0.879 

0.886 

0.893 

0.899 

0.904 

0.910 

0.916 

0.922 

0.924 

0.930 

0.935 

0.940 

0.941 

0.946 

0.950 

0.954 

0.954 

0.958 

0.962 

0.965 

0.965 

0.968 

0.971 

0.974 

0.973 

0.976 

0.978 

0.981 

0.979 

0.982 

0.984 

0.986 

0.984 

0.986 

0.988 

0.990 

0.988 

0.990 

0.991 

0:992 

0.991 

0.992 

0.994 

0.994 

0.993 

0.994 

0.995 

0.996 

0.995 

0.996 

0.997 

0.997 

0.996 

0.997 

0.998 

0.998 

0.997 

0.998 

0.998 

0.999 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

0.235 

0.236 

0.236 

0.237 

0.300 

0.302 

0.304 

0.306 

0.369 

0.373 

0.376 

0.379 

0.441 

0.446 

0.450 

0.455 

0.513 

0.519 

0.525 

0.530 

0.582 

0.589 

0.596 

0.602 

0.648 

0.655 

0.662 

0.669 

0.707 

0.715 

0.722 

0.729 

0.760 

0.767 

0.775 

0.782 

0.806 

0.813 

0.820 

0.827 

0.845 

0.852 

0.859 

0.865 

0.878 

0.884 

0.890 

0.896 

0.905 

0.911 

0.916 

0.921 

0.927 

0.932 

0.936 

0.941 

0.944 

0.948 

0.952 

0.956 

0.958 

0.961 

0.965 

0.968 

0.969 

0.971 

0.974 

0.976 

0.977 

0.979 

0.981 

0.983 

0.983 

0.985 

0.986 

0.988 

0.987 

0.989 

0.990 

0.991 

0.991 

0.992 

0.993 

0.994 

0.993 

0.994 

0.995 

0.996 

0.995 

0.996 

0.997 

0.997 

0.997 

0.997 

0.998 

0.998 

0.998 

0.998 

0.998 

0.999 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

B-9 


probability  of  demonstrating  technical  requirement 
probability  percent  confidence 

EXPECTED  number  OF  FAILURES 

TVTR  37  38  39  40  41  ^ 


LOO 
1.05 
1.10 
1.15  ’ 

1.20 
1.25 
1.30 
1.35 
1.40 
1.45 
1.50 
1.55 
1.60 
1.65 
1.70 
1.75 
1.80 
1.85 
1.90 
1.95 
2.00 
2.05 
2.10 
2.15 
2.20 
2.25 
2.30 
2.35 
2.40 
2.45 
2.50 
2.55 
2.60 
2.65 
2.70 
2.75 
2.80 
2.85 
2.90 
2.95 
3.00 


0.238 

0.308 

0.382 

0.459 

0.535 

0.608 

0.676 

0.736 

0.789 

0.834 

0.871 

0.901 

0.925 

0.944 

0.959 

0.970 

0.979 

0.985 

0.989 

0.992 

0.995 

0.996 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.239  I 
0.309 
0.385 
0.464 
0.541 
0.614 
0.682 
0.743 
0.795 
0.840 
0.877 
0.906 
0.930 
0.948 
0.962 
0.973 
0.980 
0.986 
0.990 
0.993 
0.995 
0.997 
0.998 
0.999 
0.999 
0.999 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 


0.240 
0.311 
0.388 
0.468 
0.546 
0.620 
0.688 
0.749 
0.802 
0.846 
0.882 
0.911 
0.934 
0.952 
0.965 
0.975 
0.982 
0.988 
0.991 
0.994 
0.996 
0.997 
0.998 
0.999 
0.999 
0.999 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
I  1.000 


0.240 

0.313 

0.391 

0.472 

0.551 

0.626 

0.695 

0.755 

0.808 

0.851 

0.887 

0.916 

0.938 

0.955 

0.968 

0.977 

0.984 

0.989 

0.992 

0.995 

0.996 

0.998 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOC 

l.OOC 


0.241 

0.315 

0.394 

0.476 

0.556 

0.632 

0.701 

0.762 

0.814 

0.857 

0.892 

0.920 

0.941 

0.958 

0.970 

0.979 

0.985 

0.990 

0.993 

0.995 

0.997 

0.998 


0.242 

0.317 

0.397 

0.480 

0.561 

0.637 

0.707 

0.768 

0.819 

0.862 

0.897 

0.924 

0.945 

0.961 

0.972 

0.981 

0.987 

0.991 

0.994 

0.996 

0.997 

0.998 


0.242 

0.318 

0.400 

0.484 

0.566 

0.643 

0.712 

0.773 

0.825 

0.867 

0.901 

0.928 

0.948 

0.963 

0.974 

0.982 

0.988 

0.992 

0.995 

0.996 

0.998 

0.998 


0.999  • 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1 

1  nnn 

1  000 

0.243 

0.320 

0.403 

0.488 

0.570 

0.648 

0.718 

0.779 

0.830 

0.872 

0.906 

0.931 

0.951 

0.966 

0.976 

0.984 

0.989 

0.993 

0.995 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOC 

l.OOC 

l.OOC 

l.OOC 

l.OOC 

l.OOC 


PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
^  WITH  70  PERCENT  CONFIDENCE 

EXPECTED  NUMBER  OF  FAILURES 


M(T)/TR  45 


46  47 


1.00 

1.05 

1.10 

1.15 

1.20 

1.25 

1.30 

1.35 

1.40 

1.45 

1.50 

1.55 

1.60 

1.65 

1.70 

1.75 

1.80 

1.85 

1.90 

1.95 

2.00 

2.05 

2.10 

2.15 

2.20 

2.25 

2.30 

2.35 

2.40 

2.45 

2.50 

2.55 

2.60 

2.65 

2.70 

2.75 

2.80 

2.85 

2.90 

2.95 

3.00 


0.244 

0.322 

0.406 

0.491 

0.575 

0.653 

0.724 

0.784 

0.835 

0.877 

0.910 

0.935 

0.954 

0.968 

0.978 

0.985 

0.990 

0.993 

0.996 

0.997 

0.998 

0.999 

0.999 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.244 

0.323 

0.408 

0.495 

0.580 

0.659 

0.729 

0.790 

0.841 

0.881 

0.914 

0.938 

0.957 

0.970 

0.980 

0.986 

0.991 

0.994 

0.996 

0.998 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.245 

0.325 

0.411 

0.499 

0.584 

0.664 

0.734 

0.795 

0.845 

0.886 

0.917 

0.941 

0.959 

0.972 

0.981 

0.987 

0.992 

0.995 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.245 

0.326 

0.414 

0.502 

0.589 

0.669 

0.739 

0.800 

0.850 

0.890 

0.921 

0.944 

0.961 

0.974 

0.982 

0.988 

0.992 

0.995 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOC 

1.000 

l.OOC 


6 

0 

OO 

0 

247 

0.247 

331 

0.332 

421 

0.424 

.513 

0.516 

.602 

0.606 

.683 

0.687 

.754 

0.759 

.814 

0.819 

.863 

0.867 

.902 

0.905 

1.931 

0.934 

).952 

0.955 

).968 

0.970 

).979 

0.980 

).986 

0.987 

3.991 

0.992 

3.994 

0.995 

3.996 

0.997 

3.998 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

mill 


B- 


BABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  70  PERCENT  CONFIDENCE 

EXPECTED  NUMBER  OF  FAILURES 


PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  70  PERCENT  CONFIDENCE 


EXPECTED  NUMBER  OF  FAILURES 


M(T)/TR 

61 

62 

63 

64 

65 

66 

67 

68 

1.00 

0.251 

0.252 

0.252 

0.252 

0.253 

0.253 

0.253 

0.254 

1.05 

0.344 

0.346 

0.347 

0.348 

0.349 

0.350 

0.352 

0.353 

1.10 

0.445 

0.447 

0.449 

0.451 

0.453 

0.455 

0.457 

0.459 

1.15 

0.545 

0.548 

0.551 

0.554 

0.557 

0.560 

0.563 

0.566 

1.20 

0.640 

0.644 

0.64*8 

0.651 

0.655 

0.658 

0.661 

0.665 

1.25 

0.725 

0.729 

0.733 

0.737 

0.740 

0.744 

0.747 

0.751 

1.30 

0.797 

0.801 

0.804 

0.808 

0.812 

0.815 

0.819 

0.822 

1.35 

0.855 

0.858 

0.862 

0.865 

0.868 

0.871 

0.874 

0.877 

1.40 

0.899 

0.902 

0.905 

0.908 

0.911 

0.913 

0.916 

0.918 

1.45 

0.932 

0.934 

0.937 

0.939 

0.941 

0.943 

0.945 

0.947 

1.50 

0.955 

0.957 

0.959 

0.961 

0.962 

0.964 

0.966 

0.967 

1.55 

0.971 

0.973 

0.974 

0.976 

0.977 

0.978 

0.979 

0.980 

1.60 

0.982 

0.983 

0.984 

0.985 

0.986 

0.987 

0.987 

0.988 

1.65 

0.989 

0.990 

0.990 

0.991 

0.992 

0.992 

0.993 

0.993 

1.70 

0.993 

0.994 

0.994 

0.995 

0.995 

0.996 

0.996 

0.996 

1.75 

0.996 

0.997 

0.997 

0.997 

0.997 

0.998 

0.998 

0.998 

1.80 

0.998 

0.998 

0.998 

0.998 

0.998 

0.999 

0.999 

0.999 

1.85 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.90 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.05 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.10 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.15 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.20 

i.ooo 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.25 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.30 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.35 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.40 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.45 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.50 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.55 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.60 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.65 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.70 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.75 

1.000 

1.000 

1.000 

1.000 

I.ooo 

1.000 

1.000 

1.000 

2.80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.85 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

3.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

B-13 


OBABILITY 


requirement 


PR 


OF  DEMONSTRATING  TECHNICAL 
WITH  70  PERCENT  CONFIDENCE 


B-14 


iagj] 


0.256 

0.363 

0.477 

0.590 

0.693 

0.780 

0.849 

0.901 

0.937 

0.962 

0.978 

0.987 

0.993 

0.996 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.257 

0.364 

0.479 

0.593 

0.696 

0.783 

0.852 

0.903 

0.939 

0.963 

0.979 

0.988 

0.993 

0.997 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.257 

0.365 

0.481 

0.595 

0.699 

0.786 

0.855 

0.906 

0.941 

0.965 

0.980 

0.989 

0.994 

0.997 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.257 

0.366 

0.483 

0.598 

0.702 

0.789 

0.857 

0.908 

0.943 

0.966 

0.980 

0.989 

0.994 

0.997 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.257 
0.367 
0.485 
0.600 
0.704 
0.792 
0.860 
0.910 
0.944 
0.967 
0.981 
0.990 
0.994 
0.997 
0.999 
0.999 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
,  1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 


0.258 

0.368 

0.486 

0.602 

0.707 

0.795 

0.862 

0.912 

0.946 

0.968 

0.982 

0.990 

0.995 

0.997 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.258 

0.369 

0.488 

0.605 

0.710 

0.797 

0.865 

0.914 

0.948 

0.969 

0.983 

0.991 

0.995 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.258 

0.370 

0.490 

0.607 

0.713 

0.800 

0.867 

0.916 

0.949 

0.971 

0.984 

0.991 

0.995 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


B-15 


OBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  70  PERCENT  CONFIDENCE 

EXPECTED  NUMBER  OF  FAILURES 

85  86  87  88  89  90  91 


92 


OBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENl 
WITH  70  PERCENT  CONFIDENCE 


ECTED  NUMBER  OF  FAILURES 


0.260 

0.379 

0.506 

0.628 

0.736 

0.823 

0.887 

0.932 

0.961 

0.979 

0.989 

0.994 

0.997 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.260 

0.379 

0.507 

0.630 

0.738 

0.825 

0.889 

0.934 

0.962 

0.979 

0.989 

0.995 

0.997 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.261 

0.380 

0.509 

0.633 

0.741 

0.827 

0.891 

0.935 

0.963 

0.980 

0.990 

0.995 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.261 

0.381 

0.510 

0.635 

0.743 

0.830 

0.893 

0.937 

0.964 

0.981 

0.990 

0.995 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.261 

0.382 

0.512 

0.637 

0.746 

0.832 

0.895 

0.938 

0.965 

0.982 

0.991 

0.995 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.261 

0.383 

0.514 

0.639 

0.748 

0.834 

0.897 

0.940 

0.966 

0.982 

0.991 

0.996 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 


0.261 

0.384 

0.515 

0.641 

0.750 

0.836 

0.899 

0.941 

0.967 

0.983 

0.991 

0.996 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 


0.262 

0.385 

0.517 

0.643 

0.753 

0.839 

0.901 

0.942 

0.968 

0.984 

0.992 

0.996 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1,000 

1.000 

1.000 

l.OOO 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  80  PERCENT  CONFIDENCE 


EXPECTED  NUMBER  OF  FAILURES 


M(T)/TR 

5  6  7  8 

LOO 

0.079  0.093  0.102  0.109 

1.05 

0.092  0.108  0.119  0.128 

1. 10 

0.105  0.124  0.138  0.148 

1.15 

0.120  0.141  0.157  0.170 

1.20 

0.135  0.159  0.178  0.193 

1.25 

0.151  0.178  0.200  0.218 

1.30 

0.168  0.198  0.222  0.243 

1.35 

0.185  0.218  0.245  0.269 

1.40 

0.203  0.239  0.269  0.295 

1.45 

0.221  0.260  0.293  0.322 

1.50 

0.240  0.282  0.318  0.349 

1.55 

0.259  0.304  0.342  0.377 

1.60 

0.278  0.326  0.367  0.404 

1.65 

0.297  0.348  0.392  0.431 

1.70 

0.316  0.370  0.417  0.458 

1.75 

0.336  0.392  0.441  0.484 

1.80 

0.355  0.414  0.465  0.510 

1.85 

0.374  0.436  0.488  0.535 

1.90 

0.393  0.457  0.511  0.559 

1.95 

0.412  0.478  0.534  0.583 

2.00 

0.431  0.498  0.556  0.606 

2.05 

0.449  0.519  0.577  0.628 

2.10 

0.468  0.538  0.598  0.649 

2.15 

0.485  0.557  0.618  0.669 

2.20 

0.503  0.576  0.637  0.689 

2.25 

0.520  0.594  0.656  0.708 

2.30 

0.537  0.612  0.674  0.725 

2.35 

0.553  0.629  0.691  0.742 

2.40 

0.569  0.645  0.707  0.758 

2.45 

0.585  0.661  0.723  0.773 

2.50 

0.600  0.676  0.738  0.787 

2.55 

0.614  0.691  0.752  0.801 

2.60 

0.629  0.705  0.765  0.814 

2.65 

0.642  0.718  0.778  0.826 

2.70 

0.656  0.731  0.791  0.837 

2.75 

0.669  0.744  0.802  0.847 

2.80 

0.681  0.756  0.813  0.857 

2.85 

0.693  0.767  0.824  0.867 

2.90 

0.705  0.778  0.834  0.875 

2.95 

0.716  0.789  0.843  0.884 

3.00 

0.727  0.799  0.852  0.891 

9 

11 

12 

0.114 

0.118 

0.121 

0.124 

0.135 

0.140 

0.145 

0.150 

0.157 

0.165 

0.172 

0.178 

0.181 

0.191 

0.200 

0.208 

0.207 

0.219 

0.230 

0.240 

0.234 

0.248 

0.261 

0.274 

0.261 

0.278 

0.294 

0.309 

0.290 

0.309 

0.327 

0.344 

0.319 

0.341 

0.361 

0.381 

0.348 

0.373 

0.395 

0.417 

0.378 

0.405 

0.429 

0.453 

0.408 

0.436 

0.463 

0.488 

0.437 

0.468 

0.496 

0.523 

0.466 

0.499 

0.529 

0.556 

0.495 

0.529 

0.560 

0.589 

0.523 

0.558 

0.590 

0.620 

0.550 

0.586 

0.620 

0.650 

0.576 

0.614 

0.647 

0.678 

0.602 

0.640 

0.674 

0.705 

0.626 

0.665 

0.699 

0.730 

0.650 

0.688 

0.723 

0.753 

0.672 

0.711 

0.745 

0.775 

0.694 

0.732 

0.766 

0.796 

0.714 

0.752 

0.786 

0.814 

0.733 

0.771 

0.804 

0.832 

0.751 

0.789 

0.821 

0.848 

0.769 

0.805 

0.836 

0.862 

0.785 

0.821 

0.851 

0.876 

0.800 

0.835 

0.864 

0.888 

0.814 

0.848 

0.876 

0.899 

0.828 

0.861 

0.887 

0.909 

0.840 

0.872 

0.898 

0.918 

0.852 

0.883 

0.907 

0.926 

0.863 

0.892 

0.916 

0.934 

0.873 

0.901 

0.923 

0.941 

0.882 

0.910 

0.931 

0.947 

0.891 

0.917 

0.937 

0.952 

0.899 

0.924 

0.943 

0.957 

0.907 

0.931 

0.948 

0.962 

0.914 

0.936 

0.953 

0.966 

0.920 

0.942 

0.958 

0.969 
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EMENT 


PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIRI 
WITH  80  PERCENT  CONFIDENCE 


M(T)/TH 


13 


EXPECTED  NUMBER  OF  FAILURES 
14  15  16  17  18 


19 


20 


1.00 

1.05 

1.10 

1.15 
1.20 

1.25 

1.30 

1.35 

1.40 

1.45 

1.50 

1.55 
1.60 

1.65 

1.70 

1.75 
1.80 

1.85 

1.90 

1.95 
2.00 
2.05 
2.10 

2.15 
2.20 

2.25 

2.30 

2.35 

2.40 

2.45 

2.50 

2.55 
2.60 

2.65 

2.70 

2.75 
2.80 

2.85 

2.90 

2.95 
3.00 


0.127 

0.129 

0.131 

0.133 

0.154 

0.158 

0.161 

0.164 

0.184 

0.189 

0.194 

0.199 

0.216 

0.223 

0.230 

0.237 

0.250 

0.260 

0.268 

0.277 

0.286 

0.297 

0.308 

0.319 

0.323 

0.336 

0.349 

0.362 

0.361 

0.376 

0.391 

0.406 

0.399 

0.416 

0.433 

0.449 

0.437 

0.456 

0.475 

0.492 

0.475 

0.496 

0.516 

0.534 

0.512 

0.534 

0.555 

0.575 

0.548 

0.571 

0.593 

0.614 

0.583 

0.607 

0.630 

0.651 

0.616 

0.641 

0.664 

0.686 

0.648 

0.673 

0.697 

0.719 

0.678 

0.703 

0.727 

0.749 

0.706 

0.732 

0.755 

0.776 

0.733 

0.758 

0.781 

0.802, 

0.758 

0.782 

0.805 

0.825 

0.781 

0.805 

0.826 

0.845 

0.802 

0.825 

0.846 

0.864 

0.821 

0.844 

0.864 

0.881 

0.839 

0.861 

0.880 

0.896 

0.856 

0.876 

0.894 

0.909 

0.871 

0.890 

0.907 

0.921 

0.884 

0.903 

0.918 

0.931 

0.896 

0.914 

0.928 

0.940 

0.907 

0.924 

0.937 

0.948 

0.917 

0.933 

0.945 

0.955 

0.926 

0.941 

0.952 

0.961 

0.935 

0.948 

0.958 

0.967 

0.942 

0.954 

0.964 

0.971 

0.948 

0.960 

0.968 

0.975 

0.954 

0.964 

0.973 

0.979 

0.959 

0.969 

0.976 

0.982 

0.964 

0.973 

0.979 

0.984 

0.968 

0.976 

0.982 

0.987 

0.972 

0.979 

0.984 

0.988 

0.975 

0.982 

0.986 

0.990 

0.978 

0.984 

0.988 

0.992 

0.135 

0.136 

0.138 

0.139 

0.168 

0.171 

0.173 

0.176 

0.204 

0.208 

0.213 

0.217 

0.243 

0.250 

0.255 

0.261 

0.285 

0.293 

0.301 

0.308 

0.329 

0.339 

0.348 

0.357 

0.374 

0.386 

0.397 

0.408 

0.419 

0.433 

0.446 

0.458 

0.465 

0.479 

0.494 

0.508 

0.509 

0.525 

0.541 

0.556 

0.553 

0.570 

0.586 

0.602 

0.594 

0.612 

0.630 

0.646 

0.634 

0.653 

0.670 

0.687 

0.671 

0.690 

0.708 

0.725 

0.706 

0.725 

0.743 

0.760 

0.739 

0.757 

0.775 

0.791 

0.769 

0.787 

0.804 

0.819 

0.796 

0.813 

0.830 

0.844 

0.820 

0.837 

0.853 

0.867 

0.843 

0.859 

0.873 

0.886 

0.863 

0.878 

0.891 

0.903 

0.880 

0.895 

0.907 

0.918 

0.896  . 

0.909 

0.921 

0.931 

0.910 

0.922 

0.933 

0.942 

0.922 

0.933 

0.943 

0.951 

0.933 

0.943 

0.952 

0.959 

0.942 

0.952 

0.959 

0.966 

0.951 

0.959 

0.966 

0.972 

0.958 

0.965 

0.971 

0.977 

0.964 

0.971 

0.976 

0.981 

0.969 

0.975 

0.980 

0.984 

0.974 

0.979 

0.983 

0.987 

0.977 

0.982 

0.986 

0.989 

0.981 

0.985 

0.988 

0.991 

0.984 

0.987 

0.990 

0.993 

0.986 

0.989 

0.992 

0.994 

0.988 

0.991 

0.993 

0.995 

0.990 

0.993 

0.994 

0.996 

0.991 

0.994 

0.995 

0.997 

0.993 

0.995 

0.996 

0.997 

0.994 

0.996 

0.997 

0.998 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  80  PERCENT  CONFIDENCE 


M(T)/TR 


21 


EXPECTED  NUMBER  OF  FAILURES 
23  24  25  26 


22 


1.00 

0.141  0.142  0.143 

1.05 

0.178  0.181  0.183 

1.10 

0.221  0.225  0.228 

1.15 

0.267  0.272  0.277 

1.20 

0.316  0.323  0.330 

1.25 

0.366  0.375  0.384 

1.30 

0.418  0.429  0.439 

1.35 

0.470  0.482  0.493 

1.40 

0.521  0.534  0.546 

1.45 

0.570  0.584  0.598 

1.50 

0.617  0.632  0.646 

1.55 

0.662  0.677  0.691 

1.60 

0.703  0.718  0.732 

1.65 

0.741  0.755  0.769 

1.70 

0.775  0.789  0.803 

1.75 

0.806  0.820  0.833 

1.80 

0.834  0.847  0.859 

1.85 

0.858  0.870  0.882 

1.90 

0.879  0.891  0.901 

1.95 

0.898  0.909  0.918 

2.00 

0.914  0.924  0.932 

2.05 

0.928  0.937  0.944 

2.10 

0.940  0.948  0.954 

2.15 

0.950  0.957  0.963 

2.20 

0.958  0.964  0.970 

2.25 

0.966  ,0.971  0.975 

2.30 

0.972  0.976  0.980 

2.35 

0.977  0.981  0.984 

2.40 

0.981  0.984  0.987 

2.45 

0.984  0.987  0.990 

2.50 

0.987  0.990  0.992 

2.55 

0.989  0.992  0.993 

2.60 

0.991  0.993  0.995 

2.65 

0.993  0.995  0.996 

2.70 

0.994  0.996  0.997 

2.75 

0.995  0.996  0.997 

2.80 

0.996  0.997  0.998 

2.85 

0.997  0.998  0.998 

2.90 

0.997  0.998  0.999 

2.95 

0.998  0.999  0.999 

3.00 

0.998  0.999  0.999 

0.144 

0.185 

0.232 

0.282 

0.336 

0.392 

0.448 

0.504 

0.558 

0.610 

0.659 

0.704 

0.746 

0.783 

0.816 

0.845 

0.870 

0.892 

0.911 

0.926 

0.940 

0.951 

0.960 

0.968 

0.974 

0.979 

0.983 

0.987 

0.989 

0.992 

0.993 

0.995 

0.996 

0.997 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 


0.145 
0.187 
0.235 
0.287 
0.343 
0.400 
0.458 
0.515 
0.570 
0.623 
0.672 
0.717 
0.758 
0.795 
0.828 
0.856 
0.880 
0.901 
0.919 
0.934 
0.947 
0.957 
0.965 
0.972  ■ 
0.978 
0.982 
0.986 
0.989 
0.991 
0.993 
0.995 
0.996 
0.997 
0.997 
0.998 
0.998 
0.999 
0.999 
0.999 
0.999 
1.000 


0.146 

0.190 

0.239 

0.292 

0.349 

0.408 

0.467 

0.525 

0.582 

0.635 

0.684 

0.730 

0.770 

0.807 

0.839 

0.866 

0.890 

0.910 

0.927 

0.941 

0.953 

0.962 

0.970 

0.976 

0.981 

0.985 

0.988 

0.991 

0.993 

0.994 

0.996 

0.997 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 


27 


0.147 

0.192 

0.242 

0.297 

0.355 

0.416 

0.476 

0.535 

0.593 

0.646 

0.696 

0.741 

0.782 

0.818 

0.849 

0.876 

0.899 

0.918 

0.934 

0.947 

0.958 

0.967 

0.974 

0.979 

0.984 

0.987 

0.990 

0.992 

0.994 

0.996 

0.997 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 


28 


0.148 

0.193 

0.245 

0.302 

0.361 

0.423 

0.485 

0.545 

0.603 

0.658 

0.708 

0.753 

0.793 

0.828 

0.859 

0.885 

0.907 

0.925 

0.940 

0.953 

0.963 

0.971 

0.977 

0.982 

0.986 

0.989 

0.992 

0.994 

0.995 

0.996 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  80  PERCENT  CONFIDENCE 

EXPECTED  NUMBER  OF  FAILURES 


M(T)/TR 


29 


30 


31 


32  33 


34 


35  36 


1.00 

1.05 

1.10 

1.15 
1.20 

1.25 

1.30 

1.35 

1.40 

1.45 

1.50 

1.55 
1.60 

1.65 

1.70 

1.75 
1.80 

1.85 

1.90 

1.95 
2.00 
2.05 
2.10 

2.15 
2.20 

2.25 

2.30 

2.35 

2.40 

2.45 

2.50 

2.55 
2.60 

2.65 

2.70 

2.75 
2.80 

2.85 

2.90 

2.95 
3.00 


0.149 

0.149 

0.150 

0.151 

0.195 

0.197 

0.199 

0.201 

0.248 

0.251 

0.254 

0.257 

0.306 

0.311 

0.315 

0.319 

0.367 

0.373 

0.379 

0.385 

0.430 

0.438 

0.445 

0.452 

0.494 

0.502 

0.510 

0.518 

0.555 

0.564 

0.574 

0.582 

0.614 

0.624 

0.633 

0.643 

0.668 

0.679 

0.689 

0.699 

0.718 

0.729 

0.739 

0.749 

0.764 

0.774 

0.784 

0.793 

0.803 

0.813 

0.823 

0.832 

0.838 

0.847 

0.856 

0.864 

0.868 

0.876 

0.884 

0.892 

0.893 

0.901 

0.908 

0.915 

0.914 

0.921 

0.927 

0.933 

0.932 

0.938 

0.943 

0.948 

0.946 

0.951 

0.956 

0.960 

0.957 

0.962 

0.966 

0.969 

0.967 

0.971 

0.974 

0.977 

0.974 

0.977 

0.980 

0.983 

0.980 

0.983 

0.985 

0.987 

0.985 

0.987 

0.989 

0.990 

0.988 

0.990 

0.991 

0.993 

0.991 

0.992 

0.994 

0.995 

0.993 

0.994 

0.995 

0.996 

0.995 

0.996 

0.997 

0.997 

0.996 

0.997 

0.997 

0.998 

0.997 

0.998 

0.998 

0.998 

0.998 

0.998 

0.999 

0.999 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

0.999 

0.999 

1.000 

1.000 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

0.151 

0.152 

0.153 

0.153 

0.202 

0.204 

0.206 

0.207 

0.260 

0.263 

0.266 

0.269 

0.324 

0.328 

0.332 

0.336 

0.390 

0.396 

0.401 

0.407 

0.459 

0.465 

0.472 

0.478 

0.526 

0.534 

0.542 

0.549 

0.591 

0.600 

0.608 

0.616 

0.652 

0.661 

0.670 

0.678 

0.708 

0.717 

0.726 

0.735 

0.758 

0.767 

0.776 

0.784 

0.802 

0.811 

0.819 

0.827 

0.840 

0.848 

0.856 

0.863 

0.872 

0.879 

0.886 

0.893 

0.899 

0.905 

0.911 

0.917 

0.921 

0.926 

0.932 

0.937 

0.938 

0.943 

0.948 

0.952 

0.953 

0.957 

0.961 

0.964 

0.964 

0.967 

0.971 

0.973 

0.973 

0.976 

0.978 

0.980 

0.979 

0.982 

0.984 

0.986 

0.985 

0.987 

0.988 

0.990 

0.989 

0.990 

0.991 

0.993 

0.992 

0.993 

0.994 

0.995 

0.994 

0.995 

.  0.996 

0.996 

0.995 

0.996 

0.997 

0.997 

0.997 

0.997 

0.998 

0.998 

0.998 

0.998 

0.998 

0.999 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

LOOO 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  80  PERCENT  CONFIDENCE 

EXPECTED  NUMBER  OF  FAILURES 

M(T)/TR  37  38  39  40  41  42  43 


1.00 

1.05 

1.10 

1.15 
1.20 

1.25 

1.30 

1.35 

1.40 

1.45 

1.50 

1.55 
1.60 

1.65 

1.70 

1.75 
1.80 
1.85 
1.90 
1.95 
2.00 
2.05 
2.10 

2.15 
2.20 

2.25 

2.30 

2.35 

2.40 

2.45 

2.50 

2.55 
2.60 

2.65 

2.70 

2.75 
2.80 
2.85 
2.90 
2.95 
3.00 


0.154 

0.209 

0.271 

0.340 

0.412 

0.485 

0.556 

0.624 

0.687 

0.743 

0.792 

0.834 

0.870 

0.899 

0.923 

0.941 

0.956 

0.967 

0.976 

0.982 

0.987 

0.991 

0.994 

0.995 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.154 

0.210 

0.274 

0.344 

0.417 

0.491 

0.563 

0.632 

0.695 

0.751 

0.800 

0.842 

0.876 

0.905 

0.928 

0.946 

0.960 

0.970 

0.978 

0.984 

0.989 

0.992 

0.994 

0.996 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.155 

0.212 

0.276 

0.347 

0.422 

0.497 

0.570 

0.639 

0.703 

0.759 

0.807 

0.848 

0.883 

0.910 

0.932 

0.949 

0.963 

0.973 

0.980 

0.986 

0.990 

0.993 

0.995 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.156 

0.213 

0.279 

0.351 

0.427 

0.503 

0.577 

0.647 

0.710 

0.766 

0.814 

0.855 

0.888 

0.915 

0.937 

0.953 

0.966 

0.975 

0.982 

0.987 

0.991 

0.994 

0.996 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 


0.156 

0.214 

0.281 

0.355 

0.432 

0.509 

0.584 

0.654 

0.718 

0.773 

0.821 

0.861 

0.894 

0.920 

0.941 

0.957 

0.969 

0.977 

0.984 

0.989 

0.992 

0.995 

0.996 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.157 

0.216 

0.284 

0.359 

0.437 

0.515 

0.591 

0.661 

0.725 

0.781 

0.828 

0.867 

0.899 

0.925 

0.945 

0.960 

0.971 

0.979 

0.986 

0.990 

0.993 

0.995 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.157 

0.217 

0.286 

0.362 

0.441 

0.521 

0.597 

0.668 

0.732 

0.787 

0.834 

0.873 

0.905 

0.929 

0.948 

0.963 

0.973 

0.981 

0.987 

0.991 

0.994 

0.996 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.157 

0.218 

0.289 

0.366 

0.446 

0.526 

0.604 

0.675 

0.739 

0.794 

0.841 

0.879 

0.909 

0.933 

0.952 

0.965 

0.976 

0.983 

0.988 

0.992 

0.995 

0.996 

0.998 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
PROBABILll  Y  u  ppppeNT  CONFIDENCE 


M(T)/TR 


1.00 

1.05 

1.10 

1.15 

1.20 

1.25 

1.30 

1.35 

1.40 

1.45 

1.50 

1.55 

1.60 

1.65 

1.70 

1.75 

1.80 

1.85 

1.90 

1.95 

2.00 

2.05 

2.10 

2.15 

2.20 

2.25 

2.30 

2.35 

2.40 

2.45 

2.50 

2.55 

2.60 

2.65 

2.70 

2.75 

2.80 

2.85 

2.90 

2.95 

3.00 


EXPECTED  NUMBER  OF  FAILURES 
53  54  55  56  57  58 


0.161 

0.230 

0.309 

0.396 

0.486 

0.574 

0.656 

0.730 

0.793 

0.845 

0.887 

0.919 

0.943 

0.961 

0.974 

0.983 

0.989 

0.993 

0.995 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.161 

0.231 

0.311 

0.399 

0.490 

0.579 

0.662 

0.735 

0.798 

0.850 

0.891 

0.922 

0.946 

0.963 

0.975 

0.984 

0.990 

0.993 

0.996 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.162 
0.232 
0.313 
0.403 
0.494 
0.584 
0.667 
0.740 
0.803 
0.854 
0.895 
0.926 
0.949 
0.965 
0.977 
0.985 
0.990 
0.994 
0.996 
0.998 
0.999 
0.999 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 


0.162 

0.233 

0.316 

0.406 

0.498 

0.589 

0.672 

0.746 

0.808 

0.859 

0.899 

0.929 

0.951 

0.967 

0.979 

0.986 

0.991 

0.994 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 


1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.162 

0.234 

0.318 

0.409 

0.502 

0.593 

0.677 

0.751 

0.813 

0.863 

0.902 

0.932 

0.954 

0.969 

0.980 

0.987 

0.992 

0.995 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 


1.000 
1.000 
1.000 
1. 000 
1.000 
1.000 
1.000 
1.000 
l.OOO 
1.000 
1.000 
1.000 
1.000 


0.163 

0.235 

0.320 

0.412 

0.506 

0.598 

0.682 

0.756 

0.818 

0.867 

0.906 

0.935 

0.956 

0.971 

0.981 

0.988 

0.993 

0.995 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


1.000  .  1.000 


59  60 


0.163 

0.163 

0.236 

0.237 

0.322 

0.324 

0.415 

0.418 

0.510 

0.514 

0.603 

0.607 

0.687 

0.692 

0.761 

0.766 

0.822 

0.827 

0.872 

0.876 

0.910 

0.913 

0.938 

0.941 

0.958 

0.961 

0.973 

0.974 

0.982 

0.984 

0.989 

0.990 

0.993 

0.994 

0.996 

0.996 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  80  PERCENT  CONFIDENCE 


EXPECTED  NUMBER  OF  FAILURES 


M(T)/TF 

61 

62 

63 

64 

65 

66 

67 

68 

1.00 

0.163 

0.164 

0.164 

0.164 

0.164 

0.165 

0.165 

0.165 

1.05 

0.238 

0.239 

0.240 

0.241 

0.242 

0.243 

0.244 

0.245 

1.10 

0.326 

0.328 

0.330 

0.332 

0.334 

0.335 

0.337 

0.339 

1.15 

0.421 

0.424 

0.427 

0.430 

0.433 

0.435 

0.438 

0.441 

1.20 

0.518 

0.522 

0.526 

0.529 

0.533 

0.537 

0.540 

0.544 

1.25 

0.612 

0.616 

0.620 

0.625 

0.629 

0.633 

0.637 

0.641 

1.30 

0.697 

0.701 

0.706 

0.710 

0.715 

0.719 

0.724 

0.728 

1.35 

0.770 

0.775 

0.779 

0.784 

0.788 

0.792 

0.797 

0.801 

1.40 

0.831 

0.835 

0.840 

0.844 

0.848 

0.851 

0.855 

0.859 

1.45 

0.879 

0.883 

0.887 

0.890 

0.894 

0.897 

0.900 

0.903 

1.50 

0.916 

0.919 

0.922 

0.925 

0.928 

0.931 

0.933 

0.936 

1.55 

0.943 

0.946 

0.948 

0.950 

0.953 

0.955 

0.957 

0.959 

1.60 

0.963 

0.964 

0.966 

0.968 

0.970 

0.971 

0.973 

0.974 

1.65 

0.976 

0.977 

0.979 

0.980 

0.981 

0.982 

0.983 

0.984 

1.70 

0.985 

0.986 

0.987 

0.988 

0.988 

0.989 

0.990 

0.991 

1.75 

0.991 

0.991 

0.992 

0.992 

0.993 

0.994 

0.994 

0.994 

1.80 

0.994 

0.995 

0.995 

0.996 

0.996 

0.996 

0.997 

0.997 

1.85 

0.997 

0.997 

0.997 

0.997 

0.998 

0.998 

0.998 

0.998 

1.90 

0.998 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

1.95 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

2.00 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

2.05 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.10 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.15 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.20 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.25 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

i.poo 

2.30 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

.  2.35 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.40 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.45 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.50 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.55 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.60 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.65 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.70 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.75 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.85 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

3.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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1.00  0.165  0.166  0.166  0.166  0.166  0.167  0.167 

1.05  0.246  0.247  0.248  0.249  0.250  0.251  0.252 

1  10  0  341  0.343  0.345  0.347  0.348  0.350  0.352 

l’l5  0  444  0.447  0.449  0.452  0.455  0.457  0.460 

120  QMS  0.551  0.555  0.558  0.561  0.565  0.568 

1  25  0  645  0.649  0.653  0.657  0.661  0.665  0.668 

l’.30  0.732  0.736  0.740  0.744  0.748  0.752  0.756 

1  35  0  805  0.809  0.812  0.816  0.820  0.823  0.827 

1  40  0  862  0.866  0.869  0.873  0.876  0.879  0.882 

1  45  0  906  0.909  0.912  0.915  0.917  0.920  0.923 

I’SO  0  938  0.940  0.943  0.945  0.947  0.949  0.951 

1*55  0.960  0.962  0.964  0.965  0.967  0.968  0.970 

1  60  0  975  0.977  0.978  0.979  0.980  0.981  0.982 

l’.65  0.985  0.986  0.987  0.988  0.988  0.989  0.990 

1.70  0.991  0.992  0.992  0.993  0.993  0.994  0.994 

L75  0.995  0.995  0.996  0.996  0.996  0.997  0.997 

1.80  0.997  0.997  0.998  0.998  0.998  0.998  0.998 

1  85  0  998  0.999  0.999  0.999  0.999  0.999  0.999 

L90  0.999  0.999  0.999  0.999  0.999  0.999  1.000 

1  95  1  000  1.000  1.000  1.000  1.000  1.000  1.000 

2^00  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2  05  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2.10  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2  15  1  000  1.000  1.000  1.000  1.000  1.000  1.000 

2.20  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2  25  1  000  1.000  1.000  1.000  1.000  1.000  1.000 

2*30  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2  35  1  000  1.000  1.000  1.000  1.000  1.000  1.000 

2  40  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2.45  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2.50  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2  55  1  000  1.000  1.000  1.000  1.000  1.000  1.000 

2*60  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2  65  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2.70  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2.75  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2  80  1  000  1.000  1.000  1.000  1.000  1.000  1.000 

2.85  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2.90  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

2.95  1.000  1.000  1.000  1.000  1.000  1.000  1.000 

3.00  1.000  1.000  1.000  1.000  1.000  1.000  1.000 


0.167 

0.253 

0.354 

0.463 

0.571 

0.672 

0.759 

0.830 

0.885 

0.925 

0.953 

0.971 

0.983 

0.990 

0.995 

0.997 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  80  PERCENT  CONFIDENCE 


EXPECTED  NUMBER  OF  FAILURES 


M(T)/TR 

85 

86 

1.00 

0.169 

0.169 

1.05 

0.260 

0.261 

1.10 

0.369 

0.371 

1.15 

0.486 

0.488 

1.20 

0.600 

0.603 

1.25 

0.704 

0.707 

1.30 

0.790 

0.794 

1.35 

0.859 

0.861 

1.40 

0.909 

0.911 

1.45 

0.944 

0.945 

1.50 

0.966 

0.968 

1.55 

0.981 

0.982 

1.60 

0.989 

0.990 

1.65 

0.994 

0.995 

1.70 

0.997 

0.997 

1.75 

0.999 

0.999 

1.80 

0.999 

0.999 

1.85 

1.000 

1.000 

1.90 

1.000 

1.000 

1.95 

1.000 

1.000 

2.00 

1.000 

1.000 

2.05 

1.000 

1.000 

2.10 

1.000 

1.000 

2.15 

1.000 

1.000 

2.20 

1.000 

1.000 

2.25 

1.000 

1.000 

2.30 

1.000 

1.000 

2.35 

1.000 

1.000 

2.40 

1.000 

1.000 

2.45 

1.000 

1.000 

2.50 

1.000 

1.000 

2.55 

1.000 

1.000 

2.60 

1.000 

1.000 

2.65 

1.000 

1.000 

2.70 

1.000 

1.000 

2.75 

1.000 

1.000 

2.80 

1.000 

1.000 

2.85 

1.000 

1.000 

2.90 

1.000 

1.000 

2.95 

1.000 

1.000 

3.00 

1.000 

1.000 

87 

88 

89 

0.169 

0.169 

0.169 

0.262 

0.263 

0.264 

0.372 

0.374 

0.376 

0.491 

0.493 

0.495 

0.606 

0.609 

0.612 

0.710 

0.713 

0.716 

0.797 

0.800 

0.803 

0.864 

0.867 

0.870 

0.913 

0.916 

0.918 

0.947 

0.949 

0.950 

0.969 

0.970 

0.971 

0.982 

0.983 

0.984 

0.990 

0.991 

0.991 

0.995 

0.995 

0.996 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

91 

92 

0.170 

0.170 

0.170 

0.265 

0.265 

0.266 

0.377 

0.379 

0.380 

0.498 

0.500 

0.502 

0.615 

0.618 

0.621 

0.720 

0.723 

0.726 

0.806 

0.809 

0.812 

0.872 

0.875 

0.877 

0.920 

0.922 

0.924 

0.952 

0.953 

0.955 

0.972 

0.973 

0.974 

0.985 

0.985 

0.986 

0.992 

0.992 

0.993 

0.996 

0.996 

0.996 

0.998 

0.998 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  80  PERCENT  CONFIDENCE 


:(T)/TR 


1.00 

1.05 

1.10 

1.15 
1.20 

1.25 

1.30 

1.35 

1.40 

1.45 

1.50 

1.55 
1.60 

1.65 

1.70 

1.75 
1.80 

1.85 

1.90 

1.95 
2.00 
2.05 
2.10 

2.15 
2.20 

2.25 

2.30 

2.35 

2.40 

2.45 

2.50 

2.55 
2.60 

2.65 

2.70 

2.75 
2.80 

2.85 

2.90 

2.95 
3.00 


93 


0.170 

0.267 

0.382 

0.505 

0.624 

0.729 

0.815 

0.880 

0.926 

0.956 

0.975 

0.987 

0.993 

0.997 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


94 


EXPECTED  NUMBER  OF  FAILURES 
95  96  97  98 


99 


0.170 

0.268 

0.384 

0.507 

0.626 

0.732 

0.818 

0.882 

0.928 

0.958 

0.976 

0.987 

0.993 

0.997 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.170 

0.269 

0.385 

0.509 

0.629 

0.735 

0.820 

0.885 

0.929 

0.959 

0.977 

0.988 

0.994 

0.997 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.170 

0.269 

0.387 

0.512 

0.632 

0.738 

0.823 

0.887 

0.931 

0.960 

0.978 

0.988 

0.994 

0.997 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.171 
0.270 
0.388 
0.514 
0.635 
0.741 
0.826 
0.889 
0.933 
0.961 
0.979 
0.989 
0.994 
0.997 
0.999 
0.999 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000  . 
1.000  ‘ 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 
1.000 


0.171 

0.271 

0.390 

0.516 

0.638 

0.743 

0.828 

0.891 

0.935 

0.963 

0.980 

0.989 

0.995 

0.997 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


100 


0.171 

0.272 

0.391 

0.519 

0.640 

0.746 

0.831 

0.893 

0.936 

0.964 

0.980 

0.990 

0.995 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.171 

0.272 

0.393 

0.521 

0.643 

0.749 

0.834 

0.896 

0.938 

0.965 

0.981 

0.990 

0.995 

0.998 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  90  PERCENT  CONFIDENCE 

EXPECTED  NUMBER  OF  FAILURES 


I(T)/TR 

5 

1.00 

0.036 

1.05 

0.043 

1.10 

0.050 

1.15 

0.059 

1.20 

0.068 

1.25 

0.078 

1.30 

0.088 

1.35 

0.099 

1.40 

0.111 

1.45 

0.124 

1.50 

0.136 

1.55 

0.150 

1.60 

0.164 

1.65 

0.178 

1.70 

0.193 

1.75 

0.208 

1.80 

0.223 

1.85 

0.238 

1.90 

0.254 

1.95 

0.270 

2.00 

0.286 

2.05 

0.302 

2.10 

0.3,18 

2.15 

.0.334 

2.20 

0.351 

2.25 

0.367 

2.30 

0.383 

2.35 

0.399 

2.40 

0.415 

2.45 

0.430 

2.50 

0.446 

2.55 

0.461 

2.60 

0.476 

2.65 

0.491 

2.70 

0.505 

2.75 

0.520 

2.80 

0.534 

2.85 

0.547 

2.90 

0.561 

2.95 

0.574 

3.00 

0.587 

6 

7 

0.044 

0.049 

0.052 

0.059 

0.062 

0.070 

0.072 

0.082 

0.083 

0.095 

0.095 

0.109 

0.108 

0.124 

0.121 

0.140 

0.136 

0.156 

0.151 

0.174 

0.166 

0.192 

0.183 

0.211 

0.199 

0.231 

0.217 

0.251 

0.234 

0.271 

0.252 

0.292 

0.270 

0.313 

0.289 

0.334 

0.307 

0.355 

0.326 

0.376 

0.345 

0.398 

0.364 

0.419 

0.382 

0.439 

0.401 

0.460 

0.419 

0.480 

0.437 

0.500 

0.456 

0.520 

0.473 

0.539 

0.491 

0.557 

0.508 

0.576 

0.525 

0.593 

0.541 

0.611 

0.558 

0.627 

0.573 

0.644 

0.589 

0.659 

0.604 

0.674 

0.618 

0.689 

0.633 

0.703 

0.646 

0.717 

0.660 

0.730 

0.673 

0.742 

8 

9 

0.053 

0.055 

0.064 

0.067 

0.076 

0.081 

0.089 

0.096 

0.104 

0.113 

0.120 

0.130 

0.137 

0.150 

0.156 

0.170 

0.175 

0.192 

0.195 

0.214 

0.216 

0.238 

0.237 

0.262 

0.260 

0.287 

0.282 

0.312 

0.305 

0.337 

0.329 

0.363 

0.352 

0.389 

0.376 

0.415 

0.399 

0.440 

0.423 

0.465 

0.446 

0.490 

0.469 

0.515 

0.491 

0.539 

0.513 

0.562 

0.535 

0.585 

0.556 

0.606 

0.577 

0.628 

0.597 

0.648 

0.616 

0.668 

0.635 

0.686 

0.653 

0.705 

0.670 

0.722 

0.687 

0.738 

0.703 

0.754 

0.719 

0.769 

0.733 

0.783 

0.748 

0.796 

0.761 

0.809 

0.774 

0.821 

0.786 

0.832 

0.798 

0.842 

11 

0.057 

0.059 

0.071 

0.073 

0.085 

0.089 

0.102 

0.107 

0.120 

0.127 

0.140 

0.149 

0.161 

0.172 

0.184 

0.197 

0.208 

0.223 

0.233 

0.250 

0.258 

0.278 

0.285 

0.307 

0.312 

0.337 

0.340 

0.367 

0.368 

0.397 

0.396 

0.427 

0.424 

0.457 

0.451 

0.486 

0.479 

0.515 

0.505 

0.543 

0.532 

0.570 

0.557 

0.596 

0.582 

0.622 

0.606 

0.646 

0.629 

0.670 

0.652 

0.692 

0.673 

0.714 

0.694 

0.734 

0.713 

0.753 

0.732 

0.771 

0.749 

0.788 

0.766 

0.803 

0.782 

0.818 

0.796 

0.832 

0.810 

0.845 

0.824 

0.857 

0.836 

0.868 

0.847 

0.879 

0.858 

0.889 

0.868 

0.897 

0.878 

0.906 

12 


0.060 

0.076 

0.093 

0.112 

0.134 

0.157 

0.182 

0.209 

0.238 

0.267 

0.298 

0.329 

0.361 

0.393 

0.425 

0.456 

0.488 

0.518 

0.548 

0.578 

0.606 

0.633 

0.658 

0.683 

0.706 

0.729 

0.749 

0.769 

0.787 

0.805 

0.821 

0.835 

0.849 

0.862 

0.874 

0.885 

0.895 

0.904 

0.913 

0.920 

0.927 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  90  PERCENT  CONFIDENCE 


[(T)/TR 

13 

1.00 

0.062 

1.05 

0.078 

1.10 

0.096 

1.15 

0.117 

1.20 

0.140 

1.25 

0.165 

1.30 

0.192 

1.35 

0.221 

1.40 

0.252 

1.45 

0.284 

1.50 

0.317 

1.55 

0.350 

1.60 

0.384 

1.65 

0.418 

1.70 

0.451 

1.75 

0.485 

1.80 

0.517 

1.85 

0.549 

1.90 

0.580 

1.95 

0.610 

2.00 

0.639 

2.05 

0.666 

2.10 

0.692 

2.15 

0.716 

2.20 

0.739 

2.25 

0.761 

2.30 

0.781 

2.35 

0.800 

2.40 

0.818 

2.45 

0.834 

2.50 

0.849 

2.55 

0.863 

2.60 

0.875 

2.65 

0.887 

2.70 

0.898 

2.75 

0.907 

2.80 

0.916 

2.85 

0.924 

2.90 

0.932 

2.95 

0.938 

3.00 

0.944 

EXPECTED  NUMBER  OF  FAILURES 
L  15  16  17  18  19 


0.063 

0.080 

0.099 

0.121 

0.146 

0.173 

0.202 

0.233 

0.266 

0.300 

0.335 

0.370 

0.406 

0.442 

0.477 

0.512 

0.546 

0.578 

0.610 

0.640 

0.669 

0.696 

0.722 

0.746 

0.769 

0.790 

0.809 

0.827 

0.844 

0.859 

0.873 

0.886 

0.897 

0.908 

0.917 

0.926 

0.934 

0.941 

0.947 

0.952 

0.958 


0.064 

0.082 

0.102 

0.126 

0.152 

0.181 

0.212 

0.245 

0.280 

0.316 

0.353 

0.390 

0.427 

0.465 

0.502 

0.537 

0.572 

0.606 

0.638 

0.669 

0.697 

0.725 

0.750 

0.773 

0.795 

0.816 

0.834 

0.851 

0.866 

0.881 

0.893 

0.905 

0.915 

0.925 

0.933 

0.941 

0.947 

0.953 

0.959 

0.963 

0.968 


0.065 

0.083 

0.105 

0.130 

0.158 

0.188 

0.221 

0.256 

0.293 

0.331 

0.370 

0.409 

0.448 

0.487 

0.525 

0.562 

0.598 

0.632 

0.664 

0.695 

0.723 

0.750 

0.775 

0.798 

0.819 

0.838 

0.856 

0.872 

0.886 

0.899 

0.911 

0.921 

0.930 

0.939 

0.946 

0.953 

0.958 

0.964 

0.968 

0.972 

0.975 


0.066 

0.085 

0.108 

0.134 

0.164 

0.196 

0.230 

0.267 

0.306 

0.346 

0.387 

0.428 

0.468 

0.508 

0.548 

0.585 

0.622 

0.656 

0.689 

0.719 

0.747 

0.774 

0.798 

0.820 

0.840 

0.858 

0.875 

0.890 

0.903 

0.915 

0.925 

0.935 

0.943 

0.950 

0.957 

0.962 

0.967 

0.972 

0.975 

0.979 

0.981 


0.066 

0.087 

0.111 

0.138 

0.169 

0.203 

0.239 

0.278 

0.319 

0.361 

0.403 

0.446 

0.488 

0.529 

0.569 

0.608 

0.644 

0.679 

0.711 

0.742 

0.770 

0.795 

0.819 

0.840 

0.859 

0.876 

0.892 

0.905 

0.917 

0.928 

0.938 

0.946 

0.953 

0.960 

0.965 

0.970 

0.974 

0.978 

0.981 

0.984 

0.986 


0.067 

0.088 

0.113 

0.142 

0.174 

0.210 

0.248 

0.289 

0.331 

0.375 

0.419 

0.463 

0.507 

0.549 

0.590 

0.629 

0.666 

0.701 

0.733 

0.763 

0.790 

0.815 

0.837 

0.858 

0.876 

0.892 

0.906 

0.919 

0.930 

0.940 

0.948 

0.955 

0.962 

0.967 

0.972 

0.976 

0.980 

0.983 

0.985 

0.988 

0.989 


0.068 

0.090 

0.116 

0.146 

0.180 

0.217 

0.257 

0.300 

0.344 

0.389 

0.435 

0.480 

0.525 

0.568 

0.610 

0.649 

0.686 

0.721 

0.753 

0.782 

0.809 

0.833 

0.854 

0.873 

0.891 

0.906 

0.919 

0.930 

0.940 

0.949 

0.957 

0.963 

0.969 

0.974 

0.978 

0.981 

0.984 

0.987 

0.989 

0.991 

0.992 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  90  PERCENT  CONFIDENCE 

I  EXPECTED  NUMBER  OF  FAILURES 


M(T)/TF 

21 

22 

23 

24 

25 

26 

27 

28 

1.00 

0.068 

0.069 

0.070 

0.070 

0.071 

0.071 

0.072 

0.072 

1.05 

0.091 

0.093 

0.094 

0.095 

0.096 

0.098 

0.099 

0.100 

1.10 

0.118 

0.121 

0.123 

0.125 

0.127 

0.130 

0.132 

0.134 

1.15 

0.150 

0.153 

0.157 

0.160 

0.164 

0.167 

0.170 

0.174 

1.20 

0.185 

0;190 

0.195 

0.200 

0.205 

0.209 

0.214 

0.219 

1.25 

0.224 

0.230 

0.237 

0.243 

0.250 

0.256 

0.262 

0.269 

1.30 

0.266 

0.274 

0.282 

0.290 

0.298 

0.306 

0.314 

0.322 

1.35 

0.310 

0.320 

0.330 

0.340 

0.349 

0.359 

0.368 

0.378 

1.40 

0.356 

0.368 

0.379 

0.391 

0.402 

0.413 

0.424 

0.434 

1.45 

0.403 

0.416 

0.429 

0.442 

0.455 

0.467 

0.479 

0.491 

1.50 

0.450 

0.465 

0.479 

0.494 

0.507 

0.521 

0.534 

0.547 

1.55 

0.497 

0.513 

0.528 

0.544 

0.558 

0.573 

0.587 

0.600 

1.60 

0.542 

0.560 

0.576 

0.592 

0.607 

0.622 

0.637 

0.650 

1.65 

0.587 

0.604 

0.621 

0.638 

0.654 

0.669 

0.683 

0.697 

1.70 

0.629 

0.647 

0.664 

0.681 

0.697 

0.712 

0.726 

0.740 

1.75 

0.668 

0.687 

0.704 

0.721 

0.736 

0.751 

0.765 

0.779 

1.80 

0.706 

0.724 

0.741 

0.757 

0.772 

0.787 

0.800 

0.813 

1.85 

0.740 

0.758 

0.774 

0.790 

0.805 

0.818 

0.831 

0.843 

1.90 

0.771 

0.789 

0.805 

0.820 

0.833 

0.846 

0.858 

0.869 

1.95 

0.800 

0.816 

0.832 

0.846 

0.859 

0.871 

0.882 

0.892 

2.00 

0.826 

0.841 

0.856 

0.869 

0.881 

0.892 

0.902 

0.911 

2.05 

0.849 

0.864 

0.877 

0.889 

0.900 

0.910 

0.919 

0.927 

2.10 

0.869 

0.883 

0.896 

0.907 

0.917 

0.926 

0.934 

0.941 

2.15 

0.888 

0.900 

0.912 

0.922 

0.931 

0.939 

0.946 

0.952 

2.20 

0.904 

0.915 

0.926 

0.935 

0.943 

0.950 

0.956 

0.962 

2.25 

0.918 

0.928 

0.938 

0.946 

0.953 

0.959 

0.965 

0.969 

2.30 

0.930 

0.940 

0.948 

0.955 

0.961 

0.967 

0.972 

0.976 

2.35 

0.940 

0.949 

0.957 

0.963 

0.968 

0.973 

0.977 

0.981 

2.40 

0.950 

0.957 

0.964 

0.970 

'  0.974 

0.978 

0.982 

0.985 

2.45 

0.957 

0.964 

0.970 

0.975 

0.979 

0.983 

0.985 

0.988 

2.50 

0.964 

0.970 

0.975 

0.980 

0.983 

0.986 

0.988 

0.990 

2.55 

0.970 

0.975 

0.980 

0.983 

0.986 

0.989 

0.991 

0.993 

2.60 

0.975 

0.979 

0.983 

0.986 

0.989 

0.991 

0.993 

0.994 

2.65 

0.979 

0.983 

0.986 

0.989 

0.991 

0.993 

0.994 

0.995 

2.70 

0.982 

0.986 

0.989 

0.991 

0.993 

0.994 

0.996 

0.996 

2.75 

0.985 

0.988 

0.991 

0.993 

0.994 

0.996 

0.996 

0.997 

2.80 

0.988 

0.990 

0.992 

0.994 

0.995 

0.996 

0.997 

0.998 

2.85 

0.990 

0.992 

0.994 

0.995 

0.996 

0.997 

0.998 

0.998 

2.90 

0.991 

0.993 

0.995 

0.996 

0.997 

0.998 

0.998 

0.999 

2.95 

0.993 

0.995 

0.996 

0.997 

0.998 

0.998 

0.999 

0.999 

3.00 

0.994 

0.996 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
PROBABILI  ^  percent  confidence 

EXPECTED  NUMBER  OF  FAILURES 
?Q  30  31  32  33  34  35 


1.00 

1.05 

1.10 

1.15 

1.20 

1.25 

1.30 

1.35 

1.40 

1.45 

1.50 

1.55 

1.60 

1.65 

1.70 

1.75 

1.80 

1.85 

1.90 

1.95 

2.00 

2.05 

2.10 

2.15 

2.20 

2.25 

2.30 

2.35 

2.40 

2.45 

2.50 

2.55 

2.60 

2.65 

2.70 

2.75 

2.80 

2.85 

2.90 

2.95 

3.00 


36 


0.072 
0.101 
0.136 
0.177 
0.223 
0.275 
0.329 
0.387 
0.445 
0.503 
0.559 
0.613 
0.664 
0.711 
0.753 
0.791 
0.825 
0.854 
0.880 
0.901 
0.919 
0.935 
0.947 
0.958 
0.967 
0.973 
0.979 
0.984 
0.987 
0.990 
0.992 
0.994 
0.995 
0.996 
0.997 
0.998 
0.998 
0.999 
0.999 
0.999 
0.999 


0.073 
0.102 
0.138 
0.180 
0.228 
0.281 
0.337 
0.396 
0.455 
0.514 
0.571 
0.626 
0.677 
0.723 
0.766 
0.803 
0.836 
0.865 
0.889 
0.910 
0.927 
0.941 
0.953 
0.963 
0.971 
0.977 
0.982 
0.986 
0.989 
0.992 
0.994 
0.995 
0.996 
0.997 
0.998 
0.998 
0.999 
0.999 
0.999 
0.999 
1.000 


0.073 
0.103 
0.140 
0.183 
0.232 
0.287 
0.344 
0.404 
0.465 
0.525 
0.583 
0.638 
0.689 
0.736 
0.778 
0.815 
0.847 
0.874 
0.898 
0.918 
0.934 
0.947 
0.958 
0.967 
0.974 
0.980 
0.985 
0.988 
0.991 
0.993 
0.995 
0.996 
0.997 
0.998 
0.998 
0.999 
0.999 
0.999 
0.999 
1.000 
1.000 


0.074 
0.104 
0.142 
0.186 
0.237 
0.292 
0.352 
0.413 
0.475 
0.536 
0.595 
0.650 
0.701 
0.748 
0.789 
0.825 
0.857 
0.883 
0.906 
0.925 
0.940 
0.953 
0.963 
0.971 
0.978 
0.983 
0.987 
0.990 
0.992 
0.994 
0.996 
0.997 
0.998 
0.998 
0.999 
0.999 
0.999 
0.999 
1.000 
1.000 
1.000 


0.074 
0.105 
0.144 
0.189 
0.241 
0.298 
0.359 
0.422 
0.485 
0.547 
0.606 
0.662 
0.713 
0.759 
0.800 
0.835 
0.866 
0.892 
0.913 
0.931 
0.946 
0.958 
0.967 
0.975 
0.981 
0.985 
0.989 
0.992 
0.994 
0.995 
0.996 
0.997 
0.998 
0.999 
0.999 
0.999 
0.999 
1.000 
1.000 
1.000 
1.000 


0.074 

0.106 

0.146 

0.192 

0.246 

0.304 

0.366 

0.430 

0.495 

0.557 

0.617 

0.673 

0.724 

0.770 

0.810 

0.845 

0.875 

0.900 

0.920 

0.937 

0.951 

0.962 

0.971 

0.978 

0.983 

0.987 

0.990 

0.993 

0.995 

0.996 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 


0.075 

0.107 

0.147 

0.195 

0.250 

0.310 

0.373 

0.439 

0.504 

0.568 

0.628 

0.684 

0.735 

0.780 

0.820 

0.854 

0.883 

0.907 

0.927 

0.943 

0.956 

0.966 

0.974 

0.980 

0.985 

0.989 

0.992 

0.994 

0.996 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.075 

0.108 

0.149 

0.198 

0.254 

0.315 

0.380 

0.447 

0.513 

0.578 

0.638 

0.695 

0.745 

0.790 

0.829 

0.863 

0.891 

0.914 

0.933 

0.948 

0.960 

0.970 

0.977 

0.983 

0.987 

0.991 

0.993 

0.995 

0.996 

0.997 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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probability  of  demonstrating  technical  requirement 
PROB ABlEl  1  ^  percent  confidence 


expected  number  of  failures 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  90  PERCENT  CONFIDENCE 


I(T)/TR 

45 

1.00 

0.077 

1.05 

0.116 

1.10 

0.165 

1.15 

0.224 

1.20 

0.291 

1.25 

0.364 

1.30 

0.440 

1.35 

0.516 

1.40 

0.590 

1.45 

0.659 

1.50 

0.721 

1.55 

0.776 

1.60 

0.824 

1.65 

0.863 

1.70 

0.895 

1.75 

0.921 

1.80 

0.941 

1.85 

0.957 

1.90 

0.969 

1.95 

0.978 

2.00 

0.984 

2.05 

0.989 

2.10 

0.992 

2.15 

0.995 

2.20 

0.996 

2.25 

0.998 

2.30 

0.998 

■  2.35 

0.999 

2.40 

0.999 

2.45 

1.000 

2.50 

1.000 

2.55 

1.000 

2.60 

1.000 

2.65 

1.000 

2.70 

1.000 

2.75 

1.000 

2.80 

1.000 

2.85 

1.000 

2.90 

1.000 

2.95 

1.000 

3.00 

1.000 

EXPECTED  NUMBER  OF  FAILURES 
46  47  48  49  50 


51  52 


0.077 

0.078 

0.117 

0.117 

0.167 

0.168 

0.226 

0.229 

0.295 

0.298 

0.369 

0.374 

0.446 

0.452 

0.523 

0.530 

0.598 

0.605 

0.667 

0.675 

0.729 

0.737 

0.784 

0.792 

0.831 

0.838 

0.870 

0.876 

0.901 

0.906 

0.926 

0.931 

0.945 

0.949 

0.960 

0.963 

0.972 

0.974 

0.980 

0.982 

0.986 

0.987 

0.990 

0.991 

0.993 

0.994 

0.995 

0.996 

0.997 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

0.078 

0.115 

0.170 

0.232 

0.302 

0.379 

0.458 

0.537 

0.613 

0.683 

0.745 

0.799 

0.844 

0.882 

0.911 

0.935 

0.953 

0.966 

0.976 

0.983 

0.989 

0.992 

0.995 

0.996 

0.998 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.078 

0.078 

0.119 

0.120 

0.171 

0.173 

0.234 

0.237 

0.306 

0.310 

0.384 

0.389 

0.464 

0.470 

0.544 

0.551 

•  0.620 

0.627 

0.690 

0.698 

0.752 

0.759 

0.806 

0.813 

0.851 

0.857 

0.887 

0.892 

0.916 

0.921 

0.939 

0.943 

0.956 

0.959 

0.969 

0.971 

0.978 

0.980 

0.985 

0.986 

0.990 

0.991 

0.993 

0.994 

0.995 

0.996 

0.997 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

0.078 

0.079 

0.121 

0.121 

0.175 

0.176 

0.240 

0.242 

0.314 

0.317 

0.394 

0.399 

0.476 

0.482 

0.558 

0.564 

0.635 

0.642 

0.705 

0.712 

0.767 

0.773 

0.819 

0.825 

0.863 

0.868 

0.898 

0.902 

0.925 

0.929 

0.946 

0.949 

0.962 

0.964 

0.973 

0.975 

0.982 

0.983 

0.988 

0.989 

0.992 

0.993 

0.994 

0.995 

0.996 

0.997 

0.998 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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PROBABILITY  OF  DEMONSTRATING  TECHNICAL  REQUIREMENT 
WITH  90  PERCENT  CONFIDENCE 


M(T)/TR 


53 


EXPECTED  NUMBER  OF  FAILURES 
54  55  56  57  58 


59 


60 


1.00 

1.05 

1.10 

1.15 
1.20 

1.25 

1.30 

1.35 

1.40 

1.45 

1.50 

1.55 
1.60 

1.65 

1.70 

1.75 
1.80 

1.85 

1.90 

1.95 
2.00 
2.05 
2.10 

2.15 
2.20 

2.25 

2.30 

2.35 

2.40 

2.45 

2.50 

2.55 
2.60 

2.65 

2.70 

2.75 
2.80 

2.85 

2.90 

2.95 
3.00 


0.079 

0.122 

0.178 

0.245 

0.321 

0.404 

0.488 

0.571 

0.649 

0.719 

0.780 

0.832 

0.874 

0.907 

0.933 

0.953 

0.967 

0.977 

0.985 

0.990 

0.993 

0.996 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.079 

0.123 

0.179 

0.247 

0.325 

0.408 

0.494 

0.577 

0.655 

0.726 

0.786 

0.838 

0.879 

0.912 

0.937 

0.955 

0.969 

0.979 

0.986 

0.991 

0.994 

0.996 

0.998 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.079 

0.123 

0.181 

0.250 

0.328 

0.413 

0.499 

0.584 

0.662 

0.732 

0.793 

0.843 

0.884 

0.916 

0.940 

0.958 

0.971 

0.981 

0.987 

0.992 

0.995 

0.997 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.079 

0.124 

0.182 

0.252 

0.332 

0.418 

0.505 

0.590 

0.669 

0.739 

0.799 

0.849 

0.889 

0.920 

0.943 

0.961 

0.973 

0.982 

0.988 

0.992 

0.995 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.079 

0.125 

0.184 

0.255 

0.336 

0.422 

0.511 

0.596 

0.675 

0.745 

0.805 

0.854 

0.893 

0.924 

0.947 

0.963 

0.975 

0.984 

0.989 

0.993 

0.996 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.080 

0.126 

0.185 

0.257 

0.339 

0.427 

0.516 

0.602 

0.681 

0.751 

0.811 

0.859 

0.898 

0.927 

0.950 

0.966 

0.977 

0.985 

0.990 

0.994 

0.996 

0.998 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.080 

0.126 

0.187 

0.260 

0.343 

0.432 

0.522 

0.608 

0.687 

0.757 

0.816 

0.864 

0.902 

0.931 

0.952 

0.968 

0.979 

0.986 

0.991 

0.994 

0.996 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


0.080 

0.127 

0.188 

0.262 

0.346 

0.436 

0.527 

0.614 

0.694 

0.763 

0.822 

0.869 

0.906 

0.934 

0.955 

0.970 

0.980 

0.987 

0.992 

0.995 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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61 

1.00 

0.080 

1.05 

0.128 

1.10 

0.190 

1.15 

0.265 

1.20 

0.350 

1.25 

0.441 

1.30 

0.532 

1.35 

0.620 

1.40 

0.700 

1.45 

0.769 

1.50 

0.827 

1.55 

0.874 

1.60 

0.910 

1.65 

0.938 

1.70 

0.958 

1.75 

0.972 

1.80 

0.982 

1.85 

0.988 

1.90 

0.993 

1.95 

0.995 

2.00 

0.997 

2.05 

0.998 

2.10 

0.999 

2.15 

0.999 

2.20 

1.000 

2.25 

1.000 

2.30 

1.000 

2.35 

1.000 

2.40 

1.000 

2.45 

1.000 

2.50 

1.000 

2.55 

1.000 

2.60 

1.000 

2.65 

1.000 

2.70 

1.000 

2.75 

1.000 

2.80 

1.000 

2.85 

1.000 

2.90 

1.000 

2.95 

1.000 

3.00 

1.000 

62 

63 

0.080 

0.080 

0.128 

0.129 

0.191 

0.192 

0.267 

0.270 

0.353 

0.357 

0.445 

0.450 

0.538 

0.543 

0.626 

0.631 

0.705 

0.711 

0.775 

0.780 

0.832 

0.837 

0.878 

0.883 

0.914 

0.918 

0.941 

0.944 

0.960 

0.962 

0.974 

0.975 

0.983 

0.984 

0.989 

0.990 

0.993 

0.994 

0.996 

0.996 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

64 

65 

0.081 

0.081 

0.130 

0.130 

0.194 

0.195 

0.272 

0.274 

0.360 

0.364 

0.454 

0.459 

0.548 

0.553 

0.637 

0.642 

0.717 

0.722 

0.786 

0.791 

0.842 

0.847 

0.887 

0.891 

0.921 

0.924 

0.946 

0.949 

0.964 

0.966 

0.977 

0.978 

0.985 

0.986 

0.991 

0.991 

0.994 

0.995 

0.997 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

66 


0.081 

0.131 

0.197 

0.277 

0.367 

0.463 

0.558 

0.648 

0.728 

0.796 

0.852 

0.895 

0.928 

0.951 

0.968 

0.980 

0.987 

0.992 

0.995 

0.997 

0.998 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


67  68 


0.081 

0.081 

0.131 

0.132 

0.198 

0.200 

0.279 

0.281 

0.371 

0.374 

0.467 

0.472 

0.563 

0.568 

0.653 

0.659 

0.733 

0.739 

0.801 

0.806 

0.856 

0.860 

0.899 

0.902 

0.931 

0.934 

0.954 

0.956 

0.970 

0.972 

0.981 

0.982 

0.988 

0.989 

0.993 

0.993 

0.996 

0.996 

0.997 

0.998 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

l.OOO 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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69 

70 

71 

72 

73 

74 

75 

76 

1.00 

0.081 

0.081 

0.081 

0.082 

0.082 

0.082 

0.082 

0.082 

1.05 

0.133 

0.133 

0.134 

0.135 

0.135 

0.136 

0.136 

0.137 

1.10 

0.201 

0.202 

0.204 

0.205 

0.206 

0.208 

0.209 

0.210 

1.15 

0.284 

0.286 

0.289 

0.291 

0.293 

0.295 

0.298 

0.300 

1.20 

0.377 

0.381 

0.384 

0.387 

0.391 

0.394 

0.397 

0.400 

1.25 

0.476 

0.480 

0.484 

0.488 

0.493 

0.497 

0.501 

0.505 

1.30 

0.573 

0.578 

0.583 

0.588 

0.592 

0.597 

0.602 

0.606 

1.35 

0.664 

0.669 

0.674 

0.679 

0.684 

0.689 

0.694 

0.698 

1.40 

0.744 

0.749 

0.754 

0.759 

0.763 

0.768 

0.773 

0.777 

1.45 

.0.811 

0.816 

0.820 

0.825 

0.829 

0.833 

0.837 

0.842 

1.50 

0.865 

0.869 

0.873 

0.877 

0.880 

0.884 

0.888 

0.891 

1.55 

0.906 

0.909 

0.913 

0.916 

0.919 

0.922 

0.925 

0.928 

1.60 

0.937 

0.939 

0.942 

0.944 

0.947 

0.949 

0.951 

0.953 

1.65 

0.958 

0.960 

0.962 

0.964 

0.966 

0.968 

0.969 

0.971 

1.70 

0.973 

0.975 

0.976 

0.978 

0.979 

0.980 

0.981 

0.982 

1.75 

0.983 

0.984 

0.985 

0.986 

0.987 

0.988 

0.989 

0.990 

1.80 

0.990 

0.991 

0.991 

0.992 

0.992 

0.993 

0.993 

0.994 

1.85 

0.994 

0.994 

0.995 

0.995 

0.996 

0.996 

0.996 

0.997 

1.90 

0.996 

0.997 

0.997 

0.997 

0.998 

0.998 

0.998' 

0.998 

1.95 

0.998 

0.998 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

2.00 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

2.05 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

2.10 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.15 

1.000 

1.000 

1.000 

1.000 

1.000  ■ 

1.000 

1.000 

1.000 

2.20 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.25 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.30 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.35 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.40 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.45 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.50 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.55 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.60 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.65 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.70 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.75 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.85 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

3.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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EXPECTED  NUMBER  OF  FAILURES 
78  79  80  81  82  83 


SA 


1.00 

0.082 

0.082 

1.05 

0.138 

0.138 

1.10 

0.212 

0.213 

1.15 

0.302 

0.304 

1.20 

0.404 

0.407 

1.25 

0.509 

0.513 

1.30 

0.611 

0.615 

1.35 

0.703 

0.708 

1.40 

0.782 

0.786 

1.45 

0.845 

0.849 

1.50 

0.894 

0.898 

1.55 

0.930 

0.933 

1.60 

0.955 

0.957 

1.65 

0.972 

0.974 

1.70 

0.983 

0.984 

1.75 

0.990 

0.991 

1.80 

0.994 

0.995 

1.85 

0.997 

0.997 

1.90 

mm 

1.95 

Kl 

2.00 

mm 

1.000 

2.05 

1.000 

1.000 

2.10 

1.000, 

1.000 

2.15 

1.000 

1.000 

2.20 

1.000 

1.000 

2.25 

1.000 

1.000 

2.30 

1.000 

1.000 

2.35 

1.000 

1.000 

2.40 

1.000 

1.000 

2.45 

1.000 

1.000 

2.50 

1.000 

1.000 

2.55 

1.000 

1.000 

2.60 

1.000 

1.000 

2.65 

1.000 

1.000 

2.70 

1.000 

1.000 

2.75 

1.000 

1.000 

2.80 

1.000 

1.000 

2.85 

1.000 

1.000 

2.90 

1.000 

1.000 

2.95 

1.000 

1.000 

3.00 

1.000 

1.000 

0.082 

0.082 

0.139 

0.140 

0.216 

0.217 

0.309 

0.311 

0.413 

0.416 

0.517 

0.521 

0.525 

0.620 

0.624 

0.628 

0.712 

0.717 

0.721 

0.790 

0.795 

0.799 

0.853 

0.857 

0.860 

0.901 

0.904 

0.907 

0.935 

0.938 

0.940 

0.959 

0.961 

0.963 

0.975 

0.976 

0.977 

0.985 

0.986 

0.987 

0.991 

0.992 

0.993 

0.995 

0.996 

0.996 

0.997 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

0.083 

0.083 

0.083 

0.141 

0.141 

0.142 

0.218 

0.220 

0.221 

0.313 

0.316 

0.318 

0.419 

0.422 

0.426 

0.528 

0.532 

0.536 

0.633 

0.637 

0.641 

0.725 

0.730 

0.734 

0.803 

0.807 

0.811 

0.864 

0.867 

0.871 

0.910 

0.913 

0.915 

0.942 

0.944 

0.946 

0.964 

0.966 

0.967 

0.979 

0.980 

0.981 

0.988 

0.988 

0.989 

0.993 

0.993 

0.994 

0.996 

0.996 

0.997 

0.998 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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93 

94 

95 

96 

97 

98 

99 

100 

1.00 

0.084 

0.084 

0.084 

0.084 

0.084 

0.084 

0.084 

0.084 

1.05 

0.147 

0.147 

0.148 

0.148 

0.149 

0.149 

0.150 

0.150 

1.10 

0.232 

0.234 

0.235 

0.236 

0.237 

0.239 

0.240 

0.241 

1.15 

0.337 

0.339 

0.341 

0.343 

0.345 

0.348 

0.350 

0.352 

1.20 

0.453 

0.456 

0.458 

0.461 

0.464 

0.467 

0.470 

0.473 

1.25 

0.569 

0.573 

0.576 

0.580 

0.583 

0.586 

0.590 

0.593 

1.30 

0.677 

0.681 

0.684 

0.688 

0.692 

0.695 

0.699 

0.702 

1.35 

0.769 

0.773 

0.776 

0.780 

0.783 

0.787 

0.790 

0.794 

1.40 

0.843 

0.846 

0.849 

0.852 

0.855 

0.858 

0.861 

0.864 

1.45 

0.898 

0.900 

0.903 

0.905 

0.908 

0.910 

0.913 

0.915 

1.50 

0.936 

0.938 

0.940 

0.942 

0.944 

0.946 

0.948 

0.949 

1.55 

0.962 

0.964 

0.965 

0.966 

0.968 

0.969 

0.970 

0.971 

1.60 

0.978 

0.979 

0.980 

0.981 

0.982 

0.983 

0.983 

0.984 

1.65 

0.988 

0.989 

0.989 

0.990 

0.990 

0.991 

0.991 

0.992 

1.70 

0.994 

0.994 

0.994 

0.995 

0.995 

0.995 

0.996 

0.996 

1.75 

0.997 

0.997 

0.997 

0.997 

0.998 

0.998 

0.998 

0.998 

1.80 

0.998 

0.998 

0.999 

0.999 

0.999 

0.999 

0.999 

0.999 

1.85 

0.999 

0.999 

0.999 

0.999 

0.999 

1.000 

1.000 

1.000 

1.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.05 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.10 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.15 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.20 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.25 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.30 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.35 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.40 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.45 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.50 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.55 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.60 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.65 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.70 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.75 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.80 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.85 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.90 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

2.95 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

3.00 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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appendix  c 

DERIVATIONS 


Proposition  1 . 


fobs<c  <» 


Proof. 

To  prove  this  relation,  we  use  the  equation  below  which  follows  directly  from  the 
definition  of  a  100  y  percent  lower  confidence  bound  when  failures  occur  in  a  demonstration 

test  of  length  ' 


fobs 

I 

1=0 


g-TD»/^ 


i! 


\-r 


where 


^  A  ^,(f  J. 


Let  s  be  the  function  of  x  >  0  defined  by  the  left-hand  side  of  the  equation  above  with  ^  replaced 
ty  X  Note  g“ctly  increasing  toction  of  x  >  0  since  g(x)  is  the  probability  of  obtammg 
f  or  fewer  failures  when  the  constant  configuration  under  test  has  MTBF  x. 

J  obs 

I.  First  we  shall  show  <€■=>  TR  <  i . 


Thus,  let  <  c .  Suppose  £  <  TR.  Then 


g(f)<g(TR)  - 

1=0 


i! 


1=0 


l! 


<  \-r 

which  is  a  contradiction  since  g(^)  =  !-;'•  Thus,  TR<t. 

II.  Next  we  shall  show  TR<i=>  fobs -  ^  TR<£ .  Suppose  fob,  >  ^  •  Then 
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/. 


Y^^-To.JTR  jT^JTR)^  ^  g(TR)  <  g(^)  =  1-r- 


/=0 


Since  >  c ,  this  contradicts  the  definition  of  c  (see  Equation  (5)  in  Section  2.1.2). 
Thus,/„,,  <c. 

Proposition  2. 

For  each  a<l,  T>0,  and  M(T)>0,  the  corresponding  distribution  fimction  of  Ly{N,S) 
satisfies  the  inequality 

Prob(l^(iV,5)<M(7’))  >  / 


Proof 

Let  fn,  denote  the  density  function  of  W  (defined  by  Equation  (20)  in  Section  2.1.3) 
corresponding  to  a<l,  T>0,  M(T)>0.  By  inequality  (21)  in  Section  2.1.3, 

Vxoh[L^{N,S)  <  M{t)) 


=  ]{Prob(L^ (A^,  S-,  w)  <  M(r))} f,  (w)dw 


^  rjfw(w)dw 
0 

=  r 


Proposition  3. 

For  each  a<  1 ,  T>0,  and  M(T)>0, 

Prob(i:,(Ar,5)=x)  =  0 


for  all  real  x. 

Proof 

Let  a<l,  T>0,  and  M(T)>0.  Clearly,  (A^, 5)  >  0 .  Thus,  we  need  to  consider  x  >  0 . 
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Let  L^{n,S)  denote  Ly{N,S)  conditioned  on  =  « .  As  shown  in  Appendix  A  of 
Reference  7, 


M{T) 


Zln 


where  zl  is  the  chi-square  random  variable  with  v  degrees  of  freedom. 
Thus, 

Mn(T)  ~  _ _ /?.,  I  r  Xm 


T  2 

—  Zl. 


1  ^ 

f 

[in^j 

ipci 


Then,  by  (12)  in  Section  2.1.3, 

Ar(n,S) 


f  2n  ^ 

2 

<  j,  ^ 

U.(«)  J 

[w  J 

Xi. 


i.e.. 


LXn,S) 


'  2T^ 

xL  ^ 

1/?  J 

Uk")J 

Thus, 


Prob  (Lr(n,5)  =  x)  =  Prob 


Xi.  =■ 


(n)x^ 
2T 


=  0 


It  then  follows  that. 


Prob  (Ly  (N,S)  =  x)  - 

[Prob(A  =  0)]-'  X [Prob(l, («,  5)  =  x)]Prob(A  =  n) 

n-\ 

=  0,  since  Prob  (N=0)  >  0. 


Proposition  4. 


(34) 
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Type  II  =  Prob(77?  < {N, 5)) <  1  - r  for  each  a<\  and  T >  0  where  M{t)  TR  . 

Proof. 

Let  O'  <  1  and  r  >  0  with  M{r)  =  TR  . 

Then 

Prob(r/?<I^(A^,5))= 

Prob(l^ {N,S)  =  TR)  +  Prob(77?  <L^{N,S)) 

=  Prob(TR  <  (N,  5)),  by  Proposition  3, 

=  l-Prob(i:^(7\^,5)<TR)  <  1-/,  by  Proposition  2. 

Proposition  5. 

For  a  ^owth  curve  with  parameters  (a,  T,  M(T)),  the  expected  number  of  failures  (E(N)) 
can  be  determined  by 

"  il-a)UiT) 


Proof 

The  observed  number  of  failures  by  test  duration  t,  denoted  by  N(t),  is  a  non- 
homogeneous  Poisson  process  with  N{t)=N  and  intensity  function 


/?(t) 


1 

M(t) 


This  implies  that  N  is  Poisson  distributed  with  expected  value 

T 


E(N)  =  j/7(t)dt  = 

0 


By  Equation  (18)  in  Section  2.1.3, 

E(N) 


(Mcnx^ 
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This  yields 


E{M) 


T 

^{T) 


T 

(l-a)  M(T)' 


Proposition  6. 

For  a  growth  curve  with  parameters  (a,  T,  M(T)), 
Prob  (A;  a,  T,  M(T))  = 


(i-o'T 


n=l 


Prob 


V 


where  //  A  E{n)  and  d  A  M{T)fTR  . 

Proof. 

From  (23)  in  Section  2.1.3  and  (34), 

Prob(v4;  a,T,M{T))  -  Prob(z,^  {N,S)>TR) 


=  [1  -  Prob  (N  =  0)]  '  X[Prob  (L/  (n, S)  >  TR)]  Prob  (N  =  n) 


n=l 


=  [l-Prob(A  =  0)]-' £ 


n=l 


Prob 


jL) 


>tr 


Prob  (N  =  n) 


[l-Prob(N  =  0)]"'  ^ 


n=I 


Prob 


"  xi  .  A(TR)^ 


2T 


Prob(N  =  n) 


Letting  p  A  E(N)  and  d  A  M(T)/TR, 


Prob(z4;ar,r,  A/(r))  = 


(i-z-'r'Z 


rt=l 


Prob 


zUn)  V  T  \M{T) 
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APPENDIX  D 


This  appendix  utilizes  the  notation  in  Section  4.4. 

Annex  1 

We  shall  show  the  following: 

(!)£[/, W]  =  I-e-*' 

(2) 

i=l 

To  show  (1)  observe  that  /,.(/)  is  a  random  variable  that  only  takes  on  the  values  zero  and 
one.  Thus 

£[/,(i)]  =  (o)Pr(/,(()=0)+(l)Pr(/,(()=l) 

=  Pr(/, (')=!)  = 

To  show  (2),  let  M(t)  denote  the  number  of  distinct  B-modes  that  occur  by  t.  Then 

M(0  =  ZMO 

/=! 

Thus 

Mr.A)  =  £(M(,))  .=  i£[/,W]  = 

Note  (3)  follows  from  (2)  since 

h(f,A) 

Annex  2 

Recall  A  ~  T{a,0).  Let  T'  denote  the  moment  generating  function  for  A .  Thus,  hy 
definition,  'V{x)  =  )  for  all  real  x  for  which  the  expectation  with  respect  to  A  exists.  One 


D-3 


can  show  that  T  is  defined  fox  x<^  and  4^ W  =  (l  - (see  e.g.  Mood  and  Graybill 

[9]).  We  shall  utilize  4'(x)  to  express  ,  ^{t),  h{t),  p{t),  and  0{t)  in  terms  of  K  and  the 
gamma  parameters  iz  and  We  summarize  our  results  below: 


(0  ^B,K 

(2)  //(O 

(3)  hif) 


Kj3{a-¥\)  ^  d^{t) 

+  dt 


,  ^  ^  ju.K  Ma  + 1) 

mA‘)  = 

(5)^/)  =  1-0 + 


To  show  (1),  recall  Agj^ 
A.  Thus 


where  A  =  (A  A  *:)  is  a  random  sample  from 


^B,K 


=  KE{a)  =  K 


J'F(x) 


d  X 


X  =  0 


=  K/^a  +  l) 


To  demonstrate  (2),  note  by  Annex  1 

Mt)  =  ^WnA)] 


=  E\ 


=  K-KE[e-‘^'] 


Thus 


Mt)  = 

=  a:{i-'p(-»)} 


To  derive  (3)  we  can  utilize  the  expression  for  h{f,^  in  Annex  1.  Doing  so  we  arrive  at 

h{t)  =  4^(riA)] 


=  E 


=  KE[\e-^'\ 


Note 


This  yields 


Note  by  (2)  above, 


(l  +  /?/r^ 


hit) 


K/3{a  +  \) 

(i+^/r^ 


d^)  ^  Kj3{a  +  \) 

dt  ~  {\  +  /dty^" 


Thus,  as  expected, 

^  =  £[/,(<;  A)1  =  /.(<) 

a  t 


To  obtain  (4)  we  recall  the  expression  in  (16)  of  Section  4.4.3  for  pit) : 

pif)  —  ^iy~ /^d)'^B.K'^  Md  ^if) 
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Thus  (4)  directly  follows  from  (1)  and  (3)  above. 


Finally,  recall  by  (23)  of  Section  4.4.3  we  have 

-h{t) 


40  = 


By  (1)  and  (3)  above  we  note 


^B,K 

(i+^<r 


Thus 


40  = 


2 

(i+>^0 


a+2 


A 


B.K 


=  1- (14-^0'^“ 


{a*2) 


Annex  3 

(mle-s)  for  our  finite  K  ^ 

the  AMPM,  assume  m  distinct  B.modes  first  occur  at  test  times  0<r.  <1,  S-St.  respect  y 

over  a  test  period  of  length  T.  Ut  denote  the  number  of  A-m^e  tailmes  that  oc^ur^^^^^^^ 

period  ,T.  We  shall  denote  an  estimate  of  a  model  parameter  by  placing  e  sym 
parameter.  Thus,  e.g.,  L  =  n.  /r  since  is  constant  over  test  period  T. 

Let  1  be  the  vector  of  B-mode  first  occurrence  times  (r,  Also,  let  (K)  denote 

the  set  of  po'sitive  integers  less  than  or  equal  to  K  and  let  S.  denote  the  set  of  all  subsets  of  {K) 
of  size  m  .  Then,  conditioned  on  A  =  i ,  the  likelihood  function  for  the  test  data  (».,!)  is 
where 


L{m,tjA)  =  m!  ^ 


SeS, 


isS  i&{K)-S 


-X,T 


(1) 


The  summation  in  (1)  is  over  all  the  mutually  exclusive  sets  of  exactly  m  distinct  B-modes  that 
can  occur  at  first  occurrence  times  (ti ,  •  ■  ■  >  )  • 
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Consider  the  corresponding  likelihood  random  variable 


L{m,r,A)  =  m!  ^ 


ie{K)-S 


where  A  =  (Ai, . . Ak)  is  a  random  sample  from  r(a,  P).  Denote  the  expected  value  of 
L  (m,  tT  A)  with  respect  to  A  by  L  (m,  t).  Since  the  Aj  are  independent  and  identically 
distributed  for  i  =  1, . . K.  we  have 


Z,(»i,r)  =  m! 


'K 


m 


U  E(Ae-^'‘) 


[E(e-r 


(2) 


where  A~  T  (a,  p).  We  wish  to  find  the  point  (a,  P)  that  maximizes  L(m,  t).  We  shall  denote 

A  A 

these  values  of  a  and  ^  hy  a k  and^^  ,  respectively. 


By  direct  calculation  of  E:[A^e“^],  recalling  the  form  of  density  function  given  i 

Section  4.4.2,  we  can  show 


in 


{a  +  py.^” 

>dz+l  +  p 


fiu) 


for  u<  p''  and  p  >  -(l  +  or)  •  From  (2)  and  (3)  we  obtain 


L{m,t)  =  Ar(A[^  -  l)  -(Ar  -  m  + 1) 


Let  Z  =  ln{l(m,0}-  Then  it  follows  that 


p"'{a  +  \X 


-(a^2) 


1=1 


dZ 


m 


da  a  + 1 


(3) 


(4) 


m 

-(^-m)ln(l  +  >^r)-^ln(l  +  ;^t,)  (5) 

i=l 


and 


dZ  _  m  {a^\\K-m)T  , 

—p  =  ~p  TTTr  ^  +  A 


(6) 


Treating  K  as  a  positive  real  number  we  also  obtain 
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^  =  y— - {a  +  \)\n{\^J3T)  (7) 

dK  hK-i 

In  Section  4.4  and  this  appendix  we  shall  not  use  (7)  since  we  are  only  interested  in 
obtaining  a,, ,  in  terms  of  K  and  the  test  data.  We  shall  then  hold  the  test  data  constant  and 
let  K  -^oo  to  study  the  limiting  behavior  of  our  AMPM  estimators.  Let  A(£jr,^,  AT)  and 

vk  a( maximum  likelihood  equation  for  a  is 


=  0  «  f«+lT'  =  m-'  (8) 


da  Vf.  =  Vk 


1  +  fit.  ti 


By  (6)  our  maximum  likelihood  equation  for  fi  is 


=  '^'c 


=  0  <J=>  aK  +  l  = 


fi K  ^  ^ 

{K-m)T  t,_ 

\^fi,T  ‘-'l^fiK^i 


Equating  the  expressions  for  ( + 1 1  obtained  from  (8)  and  (9)  we  arrive  at  a  linear  equation 


for  K.  Solving  for  K  we  obtain 


f,„l±4£l£_J - 

'=>  1  +  fi^  ti  '='  1  +  y^AT  A  ^^fiK^  1  +  Pk  ti 

.1  A 

^  \  +  fit^tt  X-v  fit^T 


T-t, 


For  a  given  K  and  data  set  (w,  t)  generated  over  test  period  T  we  can  solve  (10)  for  fit 
Then  we  can  use  either  (8)  or  (9)  to  obtain  ar .  Using  ( a*  we  can  estimate  all  our 


finite  K  AMPM  projection  quantities 


ies  where  Aa  =  assessed  as 
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In(ll),  d]  will  often  be  based  largely  on  engineering  judgement.  Thevalueof  rf, 

should  reflect  several  considerattons:  (1)  how 

identified;  (2)  the  nature  of  the  fix,  e.g.,  its  complexity;  (3)  past  FEF  experience  and  t  )  y 
germane  testing  (including  assembly  level  testing). 

In  practice,  we  do  not  know  the  value  of  K.  We  could  try  to  d=vdop  an  mle  for  K  based 

on  (7)  or  by  directly  maximizing  Z.  We  have  found  that  a  solution  fi.Kja  the  maximum 

likelihood  equations  (5).  (6)  and  (7)  can  be  a  saddle  point  of  iM-  “  oc-^ 
large  data  set  that  appears  to  fit  the  model  well.  We  present  graphs ^ 

rrtmctodftilure  modes  and  effects  criticality  analysis  (FMEC  A),  we  can  place  a  lower  bound 
on  K,  say  K, .  Our  experience  with  the  AMPM  is  that  if  K  is  substantially  higher  than  m ,  ^y, 
e.g..  It  2 10m ,  then  our  AMPM  projection  quantities  will  be  insensitive  to  the  value  o  . 
believe  for  a  complex  system  or  subsystem  it  will  often  be  the  case  that  Ai,  >  10m  or  at  eas  e 

"wt^ge^-lcidigr^^^ 

K  and  comparing  the  associated  projections  with  those  obtained  m  the  limit  as  K 
illustrated  for  a  data  set  in  Section  4.4.6. 

We  shall  now  consider  the  behavior  of  our  AMPM  estimators  as  AT  ->  « .  To  do  so,  let 
/^  \  be  a  sequence  satisfying  (10)  with  limit  >.  d  [0,=o).  We  shall  assume  that  such  a 

seqlenc'e'  exists  for  our  data  set  (m,<)  generated  over  [0,r].  Then  by  (10)  we  have 


In  l  +  AT  i 


J _ rnP^T  ^  Q 


(12) 


■='l  +  /7„t,. 

Recall  by  Annex  2,  =  AT  +  ')•  "here  we  previously  suppressed  the  subscript  K. 

Thus  we  shall  define  Ab.k  by 


Ab,k  a  KP\^a+\ 


lit 


(13) 


By  (8)  we  obtain 


Uk  =  KjS, 


aK  +  l 


'='  ^+/3Kh 


(14) 


Taking  the  limit  in  (14)  as  K  we  amve  at 


iflco  A  Ivm.  Ab,k  = 
= 


In  1  +  r 


(15) 


provided  /?<„  >  0 .  If  =  0 ,  then  we  can  show,  by  applying  L’Hospital’s  rule,  that  the  limit  of 
the  right  hand  side  of  (10)  goes  to  a  finite  positive  number  as  /«:  ->  oo .  This  contradiction 

(0,oo)  as  AT  ->  00  and  /?„  e  (0,oo),  we 


establishes  that  Since  Kp^ 

obtain 


A 

+  1  — ^  /ifl.o 

V  y 


flToo  =  lim  =  - 1 

a: -^00 


(16) 


We  can  now  obtain  our  limiting  AMPM  estimates  as  A:  oo .  We  first  numencally  solve 

(12)  for  P^  and  then  obtain  is,*  from  (15).  From  (16),  the  value  of  is  -1.  To  go  from  the 
finite  K  AMPM  estimate  to  the  associated  limiting  estimate,  we  first  consider  hf.  (i)  given  by  (3) 
in  Annex  2,  where  we  have  suppressed  the  subscript  K.  Motivated  by  (3),  we  define 


A 


f  - 


KP, 


flTiC+l 


V 


\aK 


+  2 


(17) 


Then 


hJt)  A  lim/iAf(t) 


i  + 


(18) 


From  (2)  in  Annex  2,  we  define 
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Mk^)  a  K  1 


V  J 


+1 


(19) 


We  can  obtain  //„(^)  more  readily  from  (18)  than  from  (19). 

A  A  /a  r  a 

A  lim =  lim  \hK{x)dx  =  flim 

=  a:^=o  J  Ja:->oo 


=  I: 


V  d!x 


From  (20),  we  can  see  that  Equation  (15)  simply  says 

M,(T)  =  m 

In  accordance  with  (5)  in  Annex  2,  we  define 

OKit)  A  \  —  ^  +  j3^t 

Then 


/ifl.OC 

A 


■In 


-1 


0^it)  A  lim^yc  =  l-fl  +  ;^„tl  = 


Finally,  from  (4)  in  Annex  2,  we  define 


( 

.  ^ 

A 

1-A. 

KJ3, 

OTac  +  I 

V 

J 

J 

^  A  \<ar/c+2 

1  + 


(20) 


(21) 


(22) 


(23) 


(24) 


From  (24)  we  have 
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Recall  in  Section  4.4.4  we  showed  our  finite  K  AMPM  converged  to  a  OTPP  in  the  sense  that 
theprocess  {A',(40<t<®}  converged  to  the  NHPP  {X.(t),OSr<oo)  as  Wealso 

noted  that  Ix  (tio<r<®)  has  the  mean  value  function  a«(<)  given  m  (4.4.4).  Wecould 
directly  derive"  parameter  estimators  for  this  NHPP.  By  so  doing,  one  can  show  that  these 
estimators  are  identical  to  the  limiting  AMPM  estimators. 
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